2024-09-11T07:50:55.042 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-11T07:50:55.057 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-11T07:50:55.148 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi/7901251 branch: wip-guits-quincy-2024-09-10-2008-quincy description: orch:cephadm/with-work/{0-distro/centos_9.stream fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rotate-keys} email: gabrioux@ibm.com first_in_suite: false job_id: '7901251' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-guits-quincy-2024-09-10-2008-quincy ceph: conf: global: mon election default strategy: 1 ms type: async mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: debug ms: 1 debug osd: 20 osd shutdown pgref assert: true flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) sha1: d0e6828a2016d48cf25ad84064e50742bb1c39b9 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} cephadm: cephadm_mode: cephadm-package install: ceph: flavor: default sha1: d0e6828a2016d48cf25ad84064e50742bb1c39b9 extra_packages: - cephadm selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 workunit: branch: wip-guits-quincy-2024-09-10-2008-quincy sha1: d0e6828a2016d48cf25ad84064e50742bb1c39b9 owner: scheduled_gabrioux@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - ceph.rgw.foo.a - node-exporter.a - alertmanager.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - grafana.a - node-exporter.b - ceph.iscsi.iscsi.a seed: 2282 sha1: d0e6828a2016d48cf25ad84064e50742bb1c39b9 sleep_before_teardown: 0 subset: 1/8 suite: orch:cephadm suite_branch: wip-guits-quincy-2024-09-10-2008-quincy suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_d0e6828a2016d48cf25ad84064e50742bb1c39b9/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: d0e6828a2016d48cf25ad84064e50742bb1c39b9 targets: smithi122.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFetlOSNfyAUeXRzr55tgJ/uEfxLZWLcvq5mjf1sP1+LluXjvXZERKqVLbh8pnzD23YdG5syKD1A0BWRp3wHPaI= smithi135.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNefVBVoRkLlSOIpL2z5sINoDM4UXTCcd/PQdR52z9G9K+UK4yE8ofPxRnbwXV53gHrggMv9FaAs2VhykiIczmg= tasks: - pexec: all: - sudo dnf remove nvme-cli -y - sudo dnf install nvmetcli nvme-cli -y - install: null - cephadm: conf: mgr: debug mgr: 20 debug ms: 1 - cephadm.shell: mon.a: - "set -ex\nfor f in osd.0 osd.1 osd.2 osd.3 osd.4 osd.5 osd.6 osd.7 mgr.y mgr.x\n\ do\n echo \"rotating key for $f\"\n K=$(ceph auth get-key $f)\n NK=\"\ $K\"\n ceph orch daemon rotate-key $f\n while [ \"$K\" == \"$NK\" ]; do\n\ \ sleep 5\n NK=$(ceph auth get-key $f)\n done\ndone\n" teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-11_06:53:43 tube: smithi user: gabrioux verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-11T07:50:55.148 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_d0e6828a2016d48cf25ad84064e50742bb1c39b9/qa; will attempt to use it 2024-09-11T07:50:55.149 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_d0e6828a2016d48cf25ad84064e50742bb1c39b9/qa/tasks 2024-09-11T07:50:55.149 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-11T07:50:55.151 INFO:teuthology.task.internal:Checking packages... 2024-09-11T07:50:55.172 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'd0e6828a2016d48cf25ad84064e50742bb1c39b9' 2024-09-11T07:50:55.172 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-11T07:50:55.172 INFO:teuthology.packaging:ref: None 2024-09-11T07:50:55.172 INFO:teuthology.packaging:tag: None 2024-09-11T07:50:55.172 INFO:teuthology.packaging:branch: wip-guits-quincy-2024-09-10-2008-quincy 2024-09-11T07:50:55.172 INFO:teuthology.packaging:sha1: d0e6828a2016d48cf25ad84064e50742bb1c39b9 2024-09-11T07:50:55.173 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-guits-quincy-2024-09-10-2008-quincy 2024-09-11T07:50:55.316 INFO:teuthology.task.internal:Found packages for ceph version 17.2.7-1623.gd0e6828a 2024-09-11T07:50:55.317 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-11T07:50:55.324 INFO:teuthology.task.internal:no buildpackages task found 2024-09-11T07:50:55.324 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-11T07:50:55.420 INFO:teuthology.task.internal:Saving configuration 2024-09-11T07:50:55.431 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-11T07:50:55.597 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-11T07:50:55.618 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi122.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi/7901251', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-11 07:45:54.217893', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFetlOSNfyAUeXRzr55tgJ/uEfxLZWLcvq5mjf1sP1+LluXjvXZERKqVLbh8pnzD23YdG5syKD1A0BWRp3wHPaI='} 2024-09-11T07:50:55.635 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi135.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi/7901251', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-11 07:45:54.219713', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNefVBVoRkLlSOIpL2z5sINoDM4UXTCcd/PQdR52z9G9K+UK4yE8ofPxRnbwXV53gHrggMv9FaAs2VhykiIczmg='} 2024-09-11T07:50:55.635 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-11T07:50:55.917 INFO:teuthology.task.internal:roles: ubuntu@smithi122.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'ceph.rgw.foo.a', 'node-exporter.a', 'alertmanager.a'] 2024-09-11T07:50:55.917 INFO:teuthology.task.internal:roles: ubuntu@smithi135.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b', 'ceph.iscsi.iscsi.a'] 2024-09-11T07:50:55.917 INFO:teuthology.run_tasks:Running task console_log... 2024-09-11T07:50:56.315 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fa44135bac0>, signals=[15]) 2024-09-11T07:50:56.315 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-11T07:50:56.576 INFO:teuthology.task.internal:Opening connections... 2024-09-11T07:50:56.576 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi122.front.sepia.ceph.com 2024-09-11T07:50:56.577 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T07:50:56.651 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi135.front.sepia.ceph.com 2024-09-11T07:50:56.652 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi135.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T07:50:56.732 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-11T07:50:57.015 DEBUG:teuthology.orchestra.run.smithi122:> uname -m 2024-09-11T07:50:57.033 INFO:teuthology.orchestra.run.smithi122.stdout:x86_64 2024-09-11T07:50:57.034 DEBUG:teuthology.orchestra.run.smithi122:> cat /etc/os-release 2024-09-11T07:50:57.091 INFO:teuthology.orchestra.run.smithi122.stdout:NAME="CentOS Stream" 2024-09-11T07:50:57.092 INFO:teuthology.orchestra.run.smithi122.stdout:VERSION="9" 2024-09-11T07:50:57.092 INFO:teuthology.orchestra.run.smithi122.stdout:ID="centos" 2024-09-11T07:50:57.092 INFO:teuthology.orchestra.run.smithi122.stdout:ID_LIKE="rhel fedora" 2024-09-11T07:50:57.092 INFO:teuthology.orchestra.run.smithi122.stdout:VERSION_ID="9" 2024-09-11T07:50:57.092 INFO:teuthology.orchestra.run.smithi122.stdout:PLATFORM_ID="platform:el9" 2024-09-11T07:50:57.092 INFO:teuthology.orchestra.run.smithi122.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-11T07:50:57.092 INFO:teuthology.orchestra.run.smithi122.stdout:ANSI_COLOR="0;31" 2024-09-11T07:50:57.092 INFO:teuthology.orchestra.run.smithi122.stdout:LOGO="fedora-logo-icon" 2024-09-11T07:50:57.092 INFO:teuthology.orchestra.run.smithi122.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-11T07:50:57.092 INFO:teuthology.orchestra.run.smithi122.stdout:HOME_URL="https://centos.org/" 2024-09-11T07:50:57.092 INFO:teuthology.orchestra.run.smithi122.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-11T07:50:57.093 INFO:teuthology.orchestra.run.smithi122.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-11T07:50:57.093 INFO:teuthology.orchestra.run.smithi122.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-11T07:50:57.093 INFO:teuthology.lock.ops:Updating smithi122.front.sepia.ceph.com on lock server 2024-09-11T07:50:57.108 DEBUG:teuthology.orchestra.run.smithi135:> uname -m 2024-09-11T07:50:57.127 INFO:teuthology.orchestra.run.smithi135.stdout:x86_64 2024-09-11T07:50:57.127 DEBUG:teuthology.orchestra.run.smithi135:> cat /etc/os-release 2024-09-11T07:50:57.185 INFO:teuthology.orchestra.run.smithi135.stdout:NAME="CentOS Stream" 2024-09-11T07:50:57.185 INFO:teuthology.orchestra.run.smithi135.stdout:VERSION="9" 2024-09-11T07:50:57.185 INFO:teuthology.orchestra.run.smithi135.stdout:ID="centos" 2024-09-11T07:50:57.185 INFO:teuthology.orchestra.run.smithi135.stdout:ID_LIKE="rhel fedora" 2024-09-11T07:50:57.185 INFO:teuthology.orchestra.run.smithi135.stdout:VERSION_ID="9" 2024-09-11T07:50:57.185 INFO:teuthology.orchestra.run.smithi135.stdout:PLATFORM_ID="platform:el9" 2024-09-11T07:50:57.185 INFO:teuthology.orchestra.run.smithi135.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-11T07:50:57.185 INFO:teuthology.orchestra.run.smithi135.stdout:ANSI_COLOR="0;31" 2024-09-11T07:50:57.185 INFO:teuthology.orchestra.run.smithi135.stdout:LOGO="fedora-logo-icon" 2024-09-11T07:50:57.185 INFO:teuthology.orchestra.run.smithi135.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-11T07:50:57.186 INFO:teuthology.orchestra.run.smithi135.stdout:HOME_URL="https://centos.org/" 2024-09-11T07:50:57.186 INFO:teuthology.orchestra.run.smithi135.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-11T07:50:57.186 INFO:teuthology.orchestra.run.smithi135.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-11T07:50:57.186 INFO:teuthology.orchestra.run.smithi135.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-11T07:50:57.186 INFO:teuthology.lock.ops:Updating smithi135.front.sepia.ceph.com on lock server 2024-09-11T07:50:57.208 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-11T07:50:57.346 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-11T07:50:57.883 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-11T07:50:57.883 DEBUG:teuthology.orchestra.run.smithi122:> test '!' -e /home/ubuntu/cephtest 2024-09-11T07:50:57.885 DEBUG:teuthology.orchestra.run.smithi135:> test '!' -e /home/ubuntu/cephtest 2024-09-11T07:50:57.903 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-11T07:50:57.976 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-11T07:50:57.977 DEBUG:teuthology.orchestra.run.smithi122:> test -z $(ls -A /var/lib/ceph) 2024-09-11T07:50:57.980 DEBUG:teuthology.orchestra.run.smithi135:> test -z $(ls -A /var/lib/ceph) 2024-09-11T07:50:58.085 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-11T07:50:58.133 INFO:teuthology.run_tasks:Running task kernel... 2024-09-11T07:50:58.241 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-11T07:50:58.241 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-11T07:50:58.241 DEBUG:teuthology.orchestra.run.smithi122:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-11T07:50:58.242 DEBUG:teuthology.orchestra.run.smithi135:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-11T07:50:58.258 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:50:58.259 DEBUG:teuthology.orchestra.run.smithi122:> uname -r 2024-09-11T07:50:58.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:50:58.260 DEBUG:teuthology.orchestra.run.smithi135:> uname -r 2024-09-11T07:50:58.316 INFO:teuthology.orchestra.run.smithi122.stdout:5.14.0-503.el9.x86_64 2024-09-11T07:50:58.316 INFO:teuthology.task.kernel:Running kernel on smithi122: 5.14.0-503.el9.x86_64 2024-09-11T07:50:58.316 DEBUG:teuthology.orchestra.run.smithi122:> sudo yum install -y kernel 2024-09-11T07:50:58.317 INFO:teuthology.orchestra.run.smithi135.stdout:5.14.0-503.el9.x86_64 2024-09-11T07:50:58.318 INFO:teuthology.task.kernel:Running kernel on smithi135: 5.14.0-503.el9.x86_64 2024-09-11T07:50:58.318 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum install -y kernel 2024-09-11T07:51:00.920 INFO:teuthology.orchestra.run.smithi135.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.2 MB 00:00 2024-09-11T07:51:03.325 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS Stream 9 - BaseOS 2.5 MB/s | 8.2 MB 00:03 2024-09-11T07:51:05.296 INFO:teuthology.orchestra.run.smithi135.stdout:CentOS Stream 9 - AppStream 6.2 MB/s | 20 MB 00:03 2024-09-11T07:51:09.872 INFO:teuthology.orchestra.run.smithi135.stdout:CentOS Stream 9 - CRB 8.5 MB/s | 6.5 MB 00:00 2024-09-11T07:51:11.348 INFO:teuthology.orchestra.run.smithi135.stdout:CentOS Stream 9 - Extras packages 72 kB/s | 19 kB 00:00 2024-09-11T07:51:11.860 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS Stream 9 - AppStream 2.7 MB/s | 20 MB 00:07 2024-09-11T07:51:12.519 INFO:teuthology.orchestra.run.smithi135.stdout:Extra Packages for Enterprise Linux 22 MB/s | 23 MB 00:01 2024-09-11T07:51:15.969 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS Stream 9 - CRB 19 MB/s | 6.5 MB 00:00 2024-09-11T07:51:17.436 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS Stream 9 - Extras packages 76 kB/s | 19 kB 00:00 2024-09-11T07:51:18.177 INFO:teuthology.orchestra.run.smithi135.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-11T07:51:18.332 INFO:teuthology.orchestra.run.smithi122.stdout:Extra Packages for Enterprise Linux 29 MB/s | 23 MB 00:00 2024-09-11T07:51:20.201 INFO:teuthology.orchestra.run.smithi135.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-11T07:51:20.201 INFO:teuthology.orchestra.run.smithi135.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-11T07:51:20.303 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2024-09-11T07:51:20.309 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-09-11T07:51:20.309 INFO:teuthology.orchestra.run.smithi135.stdout: Package Architecture Version Repository Size 2024-09-11T07:51:20.309 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-09-11T07:51:20.309 INFO:teuthology.orchestra.run.smithi135.stdout:Installing: 2024-09-11T07:51:20.309 INFO:teuthology.orchestra.run.smithi135.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-11T07:51:20.309 INFO:teuthology.orchestra.run.smithi135.stdout:Installing dependencies: 2024-09-11T07:51:20.309 INFO:teuthology.orchestra.run.smithi135.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-11T07:51:20.309 INFO:teuthology.orchestra.run.smithi135.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-11T07:51:20.310 INFO:teuthology.orchestra.run.smithi135.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-11T07:51:20.310 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-11T07:51:20.310 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction Summary 2024-09-11T07:51:20.310 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-09-11T07:51:20.310 INFO:teuthology.orchestra.run.smithi135.stdout:Install 4 Packages 2024-09-11T07:51:20.310 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-11T07:51:20.310 INFO:teuthology.orchestra.run.smithi135.stdout:Total download size: 79 M 2024-09-11T07:51:20.311 INFO:teuthology.orchestra.run.smithi135.stdout:Installed size: 126 M 2024-09-11T07:51:20.311 INFO:teuthology.orchestra.run.smithi135.stdout:Downloading Packages: 2024-09-11T07:51:20.420 INFO:teuthology.orchestra.run.smithi135.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 254 kB/s | 23 kB 00:00 2024-09-11T07:51:23.638 INFO:teuthology.orchestra.run.smithi122.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-11T07:51:23.812 INFO:teuthology.orchestra.run.smithi135.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 4.5 MB/s | 16 MB 00:03 2024-09-11T07:51:24.637 INFO:teuthology.orchestra.run.smithi135.stdout:(3/4): kernel-modules-core-5.14.0-505.el9.x86_6 6.8 MB/s | 29 MB 00:04 2024-09-11T07:51:25.636 INFO:teuthology.orchestra.run.smithi122.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-11T07:51:25.637 INFO:teuthology.orchestra.run.smithi122.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-11T07:51:25.727 INFO:teuthology.orchestra.run.smithi122.stdout:Dependencies resolved. 2024-09-11T07:51:25.738 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-11T07:51:25.738 INFO:teuthology.orchestra.run.smithi122.stdout: Package Architecture Version Repository Size 2024-09-11T07:51:25.738 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-11T07:51:25.738 INFO:teuthology.orchestra.run.smithi122.stdout:Installing: 2024-09-11T07:51:25.738 INFO:teuthology.orchestra.run.smithi122.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-11T07:51:25.738 INFO:teuthology.orchestra.run.smithi122.stdout:Installing dependencies: 2024-09-11T07:51:25.738 INFO:teuthology.orchestra.run.smithi122.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-11T07:51:25.738 INFO:teuthology.orchestra.run.smithi122.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-11T07:51:25.739 INFO:teuthology.orchestra.run.smithi122.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-11T07:51:25.739 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T07:51:25.739 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction Summary 2024-09-11T07:51:25.739 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-11T07:51:25.739 INFO:teuthology.orchestra.run.smithi122.stdout:Install 4 Packages 2024-09-11T07:51:25.739 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T07:51:25.739 INFO:teuthology.orchestra.run.smithi122.stdout:Total download size: 79 M 2024-09-11T07:51:25.740 INFO:teuthology.orchestra.run.smithi122.stdout:Installed size: 126 M 2024-09-11T07:51:25.740 INFO:teuthology.orchestra.run.smithi122.stdout:Downloading Packages: 2024-09-11T07:51:25.863 INFO:teuthology.orchestra.run.smithi135.stdout:(4/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 6.2 MB/s | 35 MB 00:05 2024-09-11T07:51:25.863 INFO:teuthology.orchestra.run.smithi135.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:51:25.863 INFO:teuthology.orchestra.run.smithi135.stdout:Total 14 MB/s | 79 MB 00:05 2024-09-11T07:51:26.082 INFO:teuthology.orchestra.run.smithi122.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 77 kB/s | 23 kB 00:00 2024-09-11T07:51:26.549 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction check 2024-09-11T07:51:26.923 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction check succeeded. 2024-09-11T07:51:26.923 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction test 2024-09-11T07:51:27.917 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction test succeeded. 2024-09-11T07:51:27.918 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction 2024-09-11T07:51:29.740 INFO:teuthology.orchestra.run.smithi122.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 4.0 MB/s | 16 MB 00:03 2024-09-11T07:51:30.368 INFO:teuthology.orchestra.run.smithi135.stdout: Preparing : 1/1 2024-09-11T07:51:31.121 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-11T07:51:31.212 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T07:51:32.998 INFO:teuthology.orchestra.run.smithi122.stdout:(3/4): kernel-modules-core-5.14.0-505.el9.x86_6 4.1 MB/s | 29 MB 00:06 2024-09-11T07:51:33.998 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T07:51:34.138 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T07:51:38.992 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T07:51:39.313 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-11T07:51:44.160 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T07:51:44.881 INFO:teuthology.orchestra.run.smithi122.stdout:(4/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 1.8 MB/s | 35 MB 00:19 2024-09-11T07:51:44.881 INFO:teuthology.orchestra.run.smithi122.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:51:44.882 INFO:teuthology.orchestra.run.smithi122.stdout:Total 4.1 MB/s | 79 MB 00:19 2024-09-11T07:51:45.555 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction check 2024-09-11T07:51:45.895 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction check succeeded. 2024-09-11T07:51:45.896 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction test 2024-09-11T07:51:46.873 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction test succeeded. 2024-09-11T07:51:46.874 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction 2024-09-11T07:51:49.312 INFO:teuthology.orchestra.run.smithi122.stdout: Preparing : 1/1 2024-09-11T07:51:50.279 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-11T07:51:50.360 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T07:51:53.132 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T07:51:53.270 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T07:51:58.086 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T07:51:58.195 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-11T07:52:03.019 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T07:52:22.461 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T07:52:22.501 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-11T07:52:22.914 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-11T07:52:22.914 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-11T07:52:22.914 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T07:52:22.914 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T07:52:23.763 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T07:52:23.763 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-11T07:52:23.764 INFO:teuthology.orchestra.run.smithi135.stdout:Installed: 2024-09-11T07:52:23.764 INFO:teuthology.orchestra.run.smithi135.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-11T07:52:23.764 INFO:teuthology.orchestra.run.smithi135.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-11T07:52:23.764 INFO:teuthology.orchestra.run.smithi135.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-11T07:52:23.764 INFO:teuthology.orchestra.run.smithi135.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-11T07:52:23.764 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-11T07:52:23.764 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2024-09-11T07:52:24.148 DEBUG:teuthology.orchestra.run.smithi135:> rpm -q kernel | sort -rV | head -n 1 2024-09-11T07:52:24.238 INFO:teuthology.orchestra.run.smithi135.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-11T07:52:24.238 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-11T07:52:24.238 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-11T07:52:24.239 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-11T07:52:24.239 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-11T07:52:24.239 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-11T07:52:24.239 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi135.front.sepia.ceph.com, path=None, version=distro) 2024-09-11T07:52:24.239 DEBUG:teuthology.orchestra.run.smithi135:> rpm -q kernel | sort -rV | head -n 1 2024-09-11T07:52:24.297 INFO:teuthology.orchestra.run.smithi135.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-11T07:52:24.298 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-11T07:52:24.298 DEBUG:teuthology.orchestra.run.smithi135:> sudo rpm -qi grub2-tools 2024-09-11T07:52:24.331 INFO:teuthology.orchestra.run.smithi135.stdout:Name : grub2-tools 2024-09-11T07:52:24.331 INFO:teuthology.orchestra.run.smithi135.stdout:Epoch : 1 2024-09-11T07:52:24.331 INFO:teuthology.orchestra.run.smithi135.stdout:Version : 2.06 2024-09-11T07:52:24.331 INFO:teuthology.orchestra.run.smithi135.stdout:Release : 82.el9 2024-09-11T07:52:24.331 INFO:teuthology.orchestra.run.smithi135.stdout:Architecture: x86_64 2024-09-11T07:52:24.331 INFO:teuthology.orchestra.run.smithi135.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-11T07:52:24.332 INFO:teuthology.orchestra.run.smithi135.stdout:Group : Unspecified 2024-09-11T07:52:24.332 INFO:teuthology.orchestra.run.smithi135.stdout:Size : 8274670 2024-09-11T07:52:24.332 INFO:teuthology.orchestra.run.smithi135.stdout:License : GPLv3+ 2024-09-11T07:52:24.332 INFO:teuthology.orchestra.run.smithi135.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-11T07:52:24.332 INFO:teuthology.orchestra.run.smithi135.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-11T07:52:24.332 INFO:teuthology.orchestra.run.smithi135.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-11T07:52:24.332 INFO:teuthology.orchestra.run.smithi135.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-11T07:52:24.332 INFO:teuthology.orchestra.run.smithi135.stdout:Packager : builder@centos.org 2024-09-11T07:52:24.332 INFO:teuthology.orchestra.run.smithi135.stdout:Vendor : CentOS 2024-09-11T07:52:24.332 INFO:teuthology.orchestra.run.smithi135.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-11T07:52:24.332 INFO:teuthology.orchestra.run.smithi135.stdout:Summary : Support tools for GRUB. 2024-09-11T07:52:24.332 INFO:teuthology.orchestra.run.smithi135.stdout:Description : 2024-09-11T07:52:24.332 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-11T07:52:24.333 INFO:teuthology.orchestra.run.smithi135.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-11T07:52:24.333 INFO:teuthology.orchestra.run.smithi135.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-11T07:52:24.333 INFO:teuthology.orchestra.run.smithi135.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-11T07:52:24.333 INFO:teuthology.orchestra.run.smithi135.stdout:hardware devices. 2024-09-11T07:52:24.333 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-11T07:52:24.333 INFO:teuthology.orchestra.run.smithi135.stdout:This subpackage provides tools for support of all platforms. 2024-09-11T07:52:24.334 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-11T07:52:24.334 INFO:teuthology.task.kernel:Updating grub on smithi135 to boot 5.14.0-505.el9.x86_64 2024-09-11T07:52:24.334 DEBUG:teuthology.orchestra.run.smithi135:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-11T07:52:25.019 INFO:teuthology.orchestra.run.smithi135.stderr:Generating grub configuration file ... 2024-09-11T07:52:26.387 INFO:teuthology.orchestra.run.smithi135.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-11T07:52:26.415 INFO:teuthology.orchestra.run.smithi135.stderr:done 2024-09-11T07:52:26.418 DEBUG:teuthology.orchestra.run.smithi135:> mktemp 2024-09-11T07:52:26.434 INFO:teuthology.orchestra.run.smithi135.stdout:/tmp/tmp.vMzqRMKv6u 2024-09-11T07:52:26.435 DEBUG:teuthology.orchestra.run.smithi135:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.vMzqRMKv6u 2024-09-11T07:52:26.501 DEBUG:teuthology.orchestra.run.smithi135:> sudo chmod 0666 /tmp/tmp.vMzqRMKv6u 2024-09-11T07:52:26.646 DEBUG:teuthology.orchestra.remote:smithi135:/tmp/tmp.vMzqRMKv6u is 6KB 2024-09-11T07:52:26.695 DEBUG:teuthology.orchestra.run.smithi135:> rm -fr /tmp/tmp.vMzqRMKv6u 2024-09-11T07:52:26.710 DEBUG:teuthology.orchestra.run.smithi135:> sudo /bin/ls /boot/loader/entries || true 2024-09-11T07:52:26.773 INFO:teuthology.orchestra.run.smithi135.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-11T07:52:26.773 INFO:teuthology.orchestra.run.smithi135.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-11T07:52:26.773 INFO:teuthology.orchestra.run.smithi135.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-11T07:52:26.773 INFO:teuthology.orchestra.run.smithi135.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-11T07:52:26.774 DEBUG:teuthology.orchestra.run.smithi135:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-11T07:52:26.929 DEBUG:teuthology.orchestra.run.smithi135:> sudo shutdown -r now 2024-09-11T07:52:41.155 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T07:52:41.195 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-11T07:52:41.594 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-11T07:52:41.594 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-11T07:52:41.594 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T07:52:41.594 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T07:52:42.450 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T07:52:42.450 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T07:52:42.450 INFO:teuthology.orchestra.run.smithi122.stdout:Installed: 2024-09-11T07:52:42.450 INFO:teuthology.orchestra.run.smithi122.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-11T07:52:42.450 INFO:teuthology.orchestra.run.smithi122.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-11T07:52:42.450 INFO:teuthology.orchestra.run.smithi122.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-11T07:52:42.450 INFO:teuthology.orchestra.run.smithi122.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-11T07:52:42.450 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T07:52:42.450 INFO:teuthology.orchestra.run.smithi122.stdout:Complete! 2024-09-11T07:52:42.788 DEBUG:teuthology.orchestra.run.smithi122:> rpm -q kernel | sort -rV | head -n 1 2024-09-11T07:52:42.866 INFO:teuthology.orchestra.run.smithi122.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-11T07:52:42.867 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-11T07:52:42.867 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-11T07:52:42.867 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-11T07:52:42.867 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-11T07:52:42.867 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-11T07:52:42.867 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi122.front.sepia.ceph.com, path=None, version=distro) 2024-09-11T07:52:42.867 DEBUG:teuthology.orchestra.run.smithi122:> rpm -q kernel | sort -rV | head -n 1 2024-09-11T07:52:42.925 INFO:teuthology.orchestra.run.smithi122.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-11T07:52:42.926 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-11T07:52:42.926 DEBUG:teuthology.orchestra.run.smithi122:> sudo rpm -qi grub2-tools 2024-09-11T07:52:42.958 INFO:teuthology.orchestra.run.smithi122.stdout:Name : grub2-tools 2024-09-11T07:52:42.958 INFO:teuthology.orchestra.run.smithi122.stdout:Epoch : 1 2024-09-11T07:52:42.958 INFO:teuthology.orchestra.run.smithi122.stdout:Version : 2.06 2024-09-11T07:52:42.958 INFO:teuthology.orchestra.run.smithi122.stdout:Release : 82.el9 2024-09-11T07:52:42.958 INFO:teuthology.orchestra.run.smithi122.stdout:Architecture: x86_64 2024-09-11T07:52:42.958 INFO:teuthology.orchestra.run.smithi122.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-11T07:52:42.958 INFO:teuthology.orchestra.run.smithi122.stdout:Group : Unspecified 2024-09-11T07:52:42.958 INFO:teuthology.orchestra.run.smithi122.stdout:Size : 8274670 2024-09-11T07:52:42.958 INFO:teuthology.orchestra.run.smithi122.stdout:License : GPLv3+ 2024-09-11T07:52:42.958 INFO:teuthology.orchestra.run.smithi122.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-11T07:52:42.958 INFO:teuthology.orchestra.run.smithi122.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-11T07:52:42.958 INFO:teuthology.orchestra.run.smithi122.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-11T07:52:42.958 INFO:teuthology.orchestra.run.smithi122.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-11T07:52:42.959 INFO:teuthology.orchestra.run.smithi122.stdout:Packager : builder@centos.org 2024-09-11T07:52:42.959 INFO:teuthology.orchestra.run.smithi122.stdout:Vendor : CentOS 2024-09-11T07:52:42.959 INFO:teuthology.orchestra.run.smithi122.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-11T07:52:42.959 INFO:teuthology.orchestra.run.smithi122.stdout:Summary : Support tools for GRUB. 2024-09-11T07:52:42.959 INFO:teuthology.orchestra.run.smithi122.stdout:Description : 2024-09-11T07:52:42.959 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T07:52:42.959 INFO:teuthology.orchestra.run.smithi122.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-11T07:52:42.959 INFO:teuthology.orchestra.run.smithi122.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-11T07:52:42.959 INFO:teuthology.orchestra.run.smithi122.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-11T07:52:42.959 INFO:teuthology.orchestra.run.smithi122.stdout:hardware devices. 2024-09-11T07:52:42.959 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T07:52:42.959 INFO:teuthology.orchestra.run.smithi122.stdout:This subpackage provides tools for support of all platforms. 2024-09-11T07:52:42.960 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-11T07:52:42.960 INFO:teuthology.task.kernel:Updating grub on smithi122 to boot 5.14.0-505.el9.x86_64 2024-09-11T07:52:42.961 DEBUG:teuthology.orchestra.run.smithi122:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-11T07:52:43.570 INFO:teuthology.orchestra.run.smithi122.stderr:Generating grub configuration file ... 2024-09-11T07:52:44.909 INFO:teuthology.orchestra.run.smithi122.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-11T07:52:44.947 INFO:teuthology.orchestra.run.smithi122.stderr:done 2024-09-11T07:52:44.950 DEBUG:teuthology.orchestra.run.smithi122:> mktemp 2024-09-11T07:52:44.966 INFO:teuthology.orchestra.run.smithi122.stdout:/tmp/tmp.W5pmKGAKyb 2024-09-11T07:52:44.966 DEBUG:teuthology.orchestra.run.smithi122:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.W5pmKGAKyb 2024-09-11T07:52:45.032 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 0666 /tmp/tmp.W5pmKGAKyb 2024-09-11T07:52:45.194 DEBUG:teuthology.orchestra.remote:smithi122:/tmp/tmp.W5pmKGAKyb is 6KB 2024-09-11T07:52:45.242 DEBUG:teuthology.orchestra.run.smithi122:> rm -fr /tmp/tmp.W5pmKGAKyb 2024-09-11T07:52:45.258 DEBUG:teuthology.orchestra.run.smithi122:> sudo /bin/ls /boot/loader/entries || true 2024-09-11T07:52:45.320 INFO:teuthology.orchestra.run.smithi122.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-11T07:52:45.320 INFO:teuthology.orchestra.run.smithi122.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-11T07:52:45.320 INFO:teuthology.orchestra.run.smithi122.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-11T07:52:45.320 INFO:teuthology.orchestra.run.smithi122.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-11T07:52:45.322 DEBUG:teuthology.orchestra.run.smithi122:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-11T07:52:45.471 DEBUG:teuthology.orchestra.run.smithi122:> sudo shutdown -r now 2024-09-11T07:52:56.944 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-11T07:52:56.944 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi135.front.sepia.ceph.com' 2024-09-11T07:52:56.945 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi135.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T07:53:15.481 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-11T07:53:15.481 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-09-11T07:53:15.482 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T07:53:56.990 DEBUG:teuthology.orchestra.remote:timed out 2024-09-11T07:54:05.999 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi135.front.sepia.ceph.com' 2024-09-11T07:54:06.001 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi135.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T07:54:06.345 DEBUG:teuthology.orchestra.run.smithi135:> true 2024-09-11T07:54:06.760 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi135.front.sepia.ceph.com' 2024-09-11T07:54:06.761 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-505.el9.x86_64"... 2024-09-11T07:54:06.761 DEBUG:teuthology.orchestra.run.smithi135:> uname -r 2024-09-11T07:54:06.775 INFO:teuthology.orchestra.run.smithi135.stdout:5.14.0-505.el9.x86_64 2024-09-11T07:54:06.775 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-11T07:54:06.775 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-11T07:54:06.776 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-11T07:54:07.776 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-11T07:54:07.776 DEBUG:teuthology.orchestra.run.smithi135:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-11T07:54:07.903 INFO:teuthology.orchestra.run.smithi135.stdout:ttyS1 2024-09-11T07:54:07.943 DEBUG:teuthology.parallel:result is None 2024-09-11T07:54:15.484 DEBUG:teuthology.orchestra.remote:timed out 2024-09-11T07:54:24.485 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-09-11T07:54:24.486 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T07:54:24.967 DEBUG:teuthology.orchestra.run.smithi122:> true 2024-09-11T07:54:25.437 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-09-11T07:54:25.438 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-505.el9.x86_64"... 2024-09-11T07:54:25.438 DEBUG:teuthology.orchestra.run.smithi122:> uname -r 2024-09-11T07:54:25.453 INFO:teuthology.orchestra.run.smithi122.stdout:5.14.0-505.el9.x86_64 2024-09-11T07:54:25.453 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-11T07:54:25.453 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-11T07:54:25.453 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-11T07:54:26.454 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-11T07:54:26.454 DEBUG:teuthology.orchestra.run.smithi122:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-11T07:54:26.540 INFO:teuthology.orchestra.run.smithi122.stdout:ttyS1 2024-09-11T07:54:26.574 DEBUG:teuthology.parallel:result is None 2024-09-11T07:54:26.574 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-11T07:54:26.582 INFO:teuthology.task.internal:Creating test directory... 2024-09-11T07:54:26.582 DEBUG:teuthology.orchestra.run.smithi122:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-11T07:54:26.584 DEBUG:teuthology.orchestra.run.smithi135:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-11T07:54:26.605 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-11T07:54:26.773 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-11T07:54:27.146 INFO:teuthology.task.internal:Creating archive directory... 2024-09-11T07:54:27.146 DEBUG:teuthology.orchestra.run.smithi122:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-11T07:54:27.149 DEBUG:teuthology.orchestra.run.smithi135:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-11T07:54:27.213 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-11T07:54:27.654 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-11T07:54:27.654 DEBUG:teuthology.orchestra.run.smithi122:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-11T07:54:27.656 DEBUG:teuthology.orchestra.run.smithi135:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-11T07:54:27.687 INFO:teuthology.orchestra.run.smithi135.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T07:54:27.699 INFO:teuthology.orchestra.run.smithi135.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T07:54:27.707 INFO:teuthology.orchestra.run.smithi122.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T07:54:27.718 INFO:teuthology.orchestra.run.smithi122.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T07:54:27.720 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-11T07:54:27.751 INFO:teuthology.task.internal:Configuring sudo... 2024-09-11T07:54:27.752 DEBUG:teuthology.orchestra.run.smithi122:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-11T07:54:27.754 DEBUG:teuthology.orchestra.run.smithi135:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-11T07:54:27.781 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-11T07:54:28.132 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-11T07:54:28.132 DEBUG:teuthology.orchestra.run.smithi122:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-11T07:54:28.135 DEBUG:teuthology.orchestra.run.smithi135:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-11T07:54:28.153 DEBUG:teuthology.orchestra.run.smithi122:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T07:54:28.234 DEBUG:teuthology.orchestra.run.smithi122:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T07:54:28.284 DEBUG:teuthology.orchestra.run.smithi122:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T07:54:28.369 DEBUG:teuthology.orchestra.run.smithi122:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T07:54:28.405 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-11T07:54:28.406 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-11T07:54:28.478 DEBUG:teuthology.orchestra.run.smithi135:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T07:54:28.523 DEBUG:teuthology.orchestra.run.smithi135:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T07:54:28.575 DEBUG:teuthology.orchestra.run.smithi135:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T07:54:28.658 DEBUG:teuthology.orchestra.run.smithi135:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T07:54:28.704 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-09-11T07:54:28.704 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-11T07:54:28.771 DEBUG:teuthology.orchestra.run.smithi122:> sudo service rsyslog restart 2024-09-11T07:54:28.773 DEBUG:teuthology.orchestra.run.smithi135:> sudo service rsyslog restart 2024-09-11T07:54:28.817 INFO:teuthology.orchestra.run.smithi122.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T07:54:28.854 INFO:teuthology.orchestra.run.smithi135.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T07:54:29.270 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-11T07:54:29.282 INFO:teuthology.task.internal:Starting timer... 2024-09-11T07:54:29.282 INFO:teuthology.run_tasks:Running task pcp... 2024-09-11T07:54:29.374 INFO:teuthology.run_tasks:Running task selinux... 2024-09-11T07:54:29.421 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-09-11T07:54:29.421 DEBUG:teuthology.orchestra.run.smithi122:> sudo service auditd rotate 2024-09-11T07:54:29.473 INFO:teuthology.orchestra.run.smithi122.stdout:Rotating logs: 2024-09-11T07:54:29.475 DEBUG:teuthology.orchestra.run.smithi135:> sudo service auditd rotate 2024-09-11T07:54:29.541 INFO:teuthology.orchestra.run.smithi135.stdout:Rotating logs: 2024-09-11T07:54:29.543 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-11T07:54:29.543 DEBUG:teuthology.orchestra.run.smithi122:> /usr/sbin/getenforce 2024-09-11T07:54:29.569 INFO:teuthology.orchestra.run.smithi122.stdout:Permissive 2024-09-11T07:54:29.569 DEBUG:teuthology.orchestra.run.smithi135:> /usr/sbin/getenforce 2024-09-11T07:54:29.598 INFO:teuthology.orchestra.run.smithi135.stdout:Permissive 2024-09-11T07:54:29.599 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi122.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi135.front.sepia.ceph.com': 'permissive'} 2024-09-11T07:54:29.599 DEBUG:teuthology.orchestra.run.smithi122:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-11T07:54:29.631 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:54:29.631 DEBUG:teuthology.orchestra.run.smithi135:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-11T07:54:29.666 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:54:29.667 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-11T07:54:29.667 DEBUG:teuthology.orchestra.run.smithi122:> sudo /usr/sbin/setenforce permissive 2024-09-11T07:54:29.736 DEBUG:teuthology.orchestra.run.smithi135:> sudo /usr/sbin/setenforce permissive 2024-09-11T07:54:29.767 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-11T07:54:30.098 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-11T07:54:30.101 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-11T07:54:30.214 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-11T07:54:30.225 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-11T07:54:30.226 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi122.front.sepia.ceph.com,smithi135.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-11T07:58:13.251 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi122.front.sepia.ceph.com'), Remote(name='ubuntu@smithi135.front.sepia.ceph.com')] 2024-09-11T07:58:13.252 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-09-11T07:58:13.253 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T07:58:13.331 DEBUG:teuthology.orchestra.run.smithi122:> true 2024-09-11T07:58:13.412 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-09-11T07:58:13.413 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi135.front.sepia.ceph.com' 2024-09-11T07:58:13.413 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi135.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T07:58:13.493 DEBUG:teuthology.orchestra.run.smithi135:> true 2024-09-11T07:58:13.568 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi135.front.sepia.ceph.com' 2024-09-11T07:58:13.568 INFO:teuthology.run_tasks:Running task clock... 2024-09-11T07:58:13.579 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-11T07:58:13.579 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-11T07:58:13.580 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T07:58:13.582 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-11T07:58:13.583 DEBUG:teuthology.orchestra.run.smithi135:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T07:58:13.609 INFO:teuthology.orchestra.run.smithi122.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-11T07:58:13.624 INFO:teuthology.orchestra.run.smithi122.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-11T07:58:13.639 INFO:teuthology.orchestra.run.smithi135.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-11T07:58:13.648 INFO:teuthology.orchestra.run.smithi122.stderr:sudo: ntpd: command not found 2024-09-11T07:58:13.653 INFO:teuthology.orchestra.run.smithi135.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-11T07:58:13.659 INFO:teuthology.orchestra.run.smithi122.stdout:506 Cannot talk to daemon 2024-09-11T07:58:13.674 INFO:teuthology.orchestra.run.smithi122.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-11T07:58:13.678 INFO:teuthology.orchestra.run.smithi135.stderr:sudo: ntpd: command not found 2024-09-11T07:58:13.688 INFO:teuthology.orchestra.run.smithi122.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-11T07:58:13.689 INFO:teuthology.orchestra.run.smithi135.stdout:506 Cannot talk to daemon 2024-09-11T07:58:13.703 INFO:teuthology.orchestra.run.smithi135.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-11T07:58:13.717 INFO:teuthology.orchestra.run.smithi135.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-11T07:58:13.746 INFO:teuthology.orchestra.run.smithi122.stderr:bash: line 1: ntpq: command not found 2024-09-11T07:58:13.749 INFO:teuthology.orchestra.run.smithi122.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T07:58:13.749 INFO:teuthology.orchestra.run.smithi122.stdout:=============================================================================== 2024-09-11T07:58:13.749 INFO:teuthology.orchestra.run.smithi122.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T07:58:13.749 INFO:teuthology.orchestra.run.smithi122.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T07:58:13.749 INFO:teuthology.orchestra.run.smithi122.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T07:58:13.749 INFO:teuthology.orchestra.run.smithi122.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T07:58:13.774 INFO:teuthology.orchestra.run.smithi135.stderr:bash: line 1: ntpq: command not found 2024-09-11T07:58:13.777 INFO:teuthology.orchestra.run.smithi135.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T07:58:13.777 INFO:teuthology.orchestra.run.smithi135.stdout:=============================================================================== 2024-09-11T07:58:13.777 INFO:teuthology.orchestra.run.smithi135.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T07:58:13.778 INFO:teuthology.orchestra.run.smithi135.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T07:58:13.778 INFO:teuthology.orchestra.run.smithi135.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T07:58:13.778 INFO:teuthology.orchestra.run.smithi135.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T07:58:13.779 INFO:teuthology.run_tasks:Running task pexec... 2024-09-11T07:58:13.966 INFO:teuthology.task.pexec:Executing custom commands... 2024-09-11T07:58:13.967 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi122.front.sepia.ceph.com 2024-09-11T07:58:13.967 DEBUG:teuthology.orchestra.run.smithi122:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-09-11T07:58:13.968 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi135.front.sepia.ceph.com 2024-09-11T07:58:13.968 DEBUG:teuthology.orchestra.run.smithi135:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-09-11T07:58:14.318 INFO:teuthology.orchestra.run.smithi122.stdout:Dependencies resolved. 2024-09-11T07:58:14.326 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2024-09-11T07:58:14.328 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-11T07:58:14.328 INFO:teuthology.orchestra.run.smithi122.stdout: Package Architecture Version Repository Size 2024-09-11T07:58:14.328 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-11T07:58:14.329 INFO:teuthology.orchestra.run.smithi122.stdout:Removing: 2024-09-11T07:58:14.329 INFO:teuthology.orchestra.run.smithi122.stdout: nvme-cli x86_64 2.9.1-6.el9 @baseos 5.3 M 2024-09-11T07:58:14.329 INFO:teuthology.orchestra.run.smithi122.stdout:Removing unused dependencies: 2024-09-11T07:58:14.329 INFO:teuthology.orchestra.run.smithi122.stdout: libnvme x86_64 1.9-3.el9 @baseos 273 k 2024-09-11T07:58:14.329 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T07:58:14.329 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction Summary 2024-09-11T07:58:14.329 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-11T07:58:14.329 INFO:teuthology.orchestra.run.smithi122.stdout:Remove 2 Packages 2024-09-11T07:58:14.329 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T07:58:14.329 INFO:teuthology.orchestra.run.smithi122.stdout:Freed space: 5.5 M 2024-09-11T07:58:14.329 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction check 2024-09-11T07:58:14.333 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction check succeeded. 2024-09-11T07:58:14.333 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction test 2024-09-11T07:58:14.339 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-09-11T07:58:14.339 INFO:teuthology.orchestra.run.smithi135.stdout: Package Architecture Version Repository Size 2024-09-11T07:58:14.339 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-09-11T07:58:14.339 INFO:teuthology.orchestra.run.smithi135.stdout:Removing: 2024-09-11T07:58:14.339 INFO:teuthology.orchestra.run.smithi135.stdout: nvme-cli x86_64 2.9.1-6.el9 @baseos 5.3 M 2024-09-11T07:58:14.339 INFO:teuthology.orchestra.run.smithi135.stdout:Removing unused dependencies: 2024-09-11T07:58:14.339 INFO:teuthology.orchestra.run.smithi135.stdout: libnvme x86_64 1.9-3.el9 @baseos 273 k 2024-09-11T07:58:14.339 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-11T07:58:14.339 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction Summary 2024-09-11T07:58:14.339 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-09-11T07:58:14.339 INFO:teuthology.orchestra.run.smithi135.stdout:Remove 2 Packages 2024-09-11T07:58:14.340 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-11T07:58:14.340 INFO:teuthology.orchestra.run.smithi135.stdout:Freed space: 5.5 M 2024-09-11T07:58:14.340 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction check 2024-09-11T07:58:14.343 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction check succeeded. 2024-09-11T07:58:14.343 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction test 2024-09-11T07:58:14.401 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction test succeeded. 2024-09-11T07:58:14.401 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction 2024-09-11T07:58:14.403 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction test succeeded. 2024-09-11T07:58:14.404 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction 2024-09-11T07:58:14.532 INFO:teuthology.orchestra.run.smithi122.stdout: Preparing : 1/1 2024-09-11T07:58:14.541 INFO:teuthology.orchestra.run.smithi135.stdout: Preparing : 1/1 2024-09-11T07:58:14.611 INFO:teuthology.orchestra.run.smithi122.stdout: Erasing : nvme-cli-2.9.1-6.el9.x86_64 1/2 2024-09-11T07:58:14.619 INFO:teuthology.orchestra.run.smithi135.stdout: Erasing : nvme-cli-2.9.1-6.el9.x86_64 1/2 2024-09-11T07:58:14.673 INFO:teuthology.orchestra.run.smithi135.stdout: Erasing : libnvme-1.9-3.el9.x86_64 2/2 2024-09-11T07:58:14.676 INFO:teuthology.orchestra.run.smithi122.stdout: Erasing : libnvme-1.9-3.el9.x86_64 2/2 2024-09-11T07:58:15.186 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: libnvme-1.9-3.el9.x86_64 2/2 2024-09-11T07:58:15.186 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/2 2024-09-11T07:58:15.221 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: libnvme-1.9-3.el9.x86_64 2/2 2024-09-11T07:58:15.221 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/2 2024-09-11T07:58:15.462 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/2 2024-09-11T07:58:15.462 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T07:58:15.463 INFO:teuthology.orchestra.run.smithi122.stdout:Removed: 2024-09-11T07:58:15.463 INFO:teuthology.orchestra.run.smithi122.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-11T07:58:15.463 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T07:58:15.463 INFO:teuthology.orchestra.run.smithi122.stdout:Complete! 2024-09-11T07:58:15.475 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/2 2024-09-11T07:58:15.475 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-11T07:58:15.476 INFO:teuthology.orchestra.run.smithi135.stdout:Removed: 2024-09-11T07:58:15.476 INFO:teuthology.orchestra.run.smithi135.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-11T07:58:15.476 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-11T07:58:15.476 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2024-09-11T07:58:16.350 INFO:teuthology.orchestra.run.smithi122.stdout:Last metadata expiration check: 0:00:49 ago on Wed 11 Sep 2024 07:57:27 AM UTC. 2024-09-11T07:58:16.368 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:00:49 ago on Wed 11 Sep 2024 07:57:27 AM UTC. 2024-09-11T07:58:16.482 INFO:teuthology.orchestra.run.smithi122.stdout:Dependencies resolved. 2024-09-11T07:58:16.483 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-11T07:58:16.483 INFO:teuthology.orchestra.run.smithi122.stdout: Package Architecture Version Repository Size 2024-09-11T07:58:16.483 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-11T07:58:16.483 INFO:teuthology.orchestra.run.smithi122.stdout:Installing: 2024-09-11T07:58:16.483 INFO:teuthology.orchestra.run.smithi122.stdout: nvme-cli x86_64 2.9.1-6.el9 baseos 906 k 2024-09-11T07:58:16.483 INFO:teuthology.orchestra.run.smithi122.stdout: nvmetcli noarch 0.7-3.el9 baseos 44 k 2024-09-11T07:58:16.483 INFO:teuthology.orchestra.run.smithi122.stdout:Installing dependencies: 2024-09-11T07:58:16.483 INFO:teuthology.orchestra.run.smithi122.stdout: libnvme x86_64 1.9-3.el9 baseos 106 k 2024-09-11T07:58:16.483 INFO:teuthology.orchestra.run.smithi122.stdout: python3-configshell noarch 1:1.1.30-1.el9 baseos 72 k 2024-09-11T07:58:16.483 INFO:teuthology.orchestra.run.smithi122.stdout: python3-kmod x86_64 0.9-32.el9 baseos 84 k 2024-09-11T07:58:16.484 INFO:teuthology.orchestra.run.smithi122.stdout: python3-urwid x86_64 2.1.2-4.el9 baseos 837 k 2024-09-11T07:58:16.484 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T07:58:16.484 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction Summary 2024-09-11T07:58:16.484 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-11T07:58:16.484 INFO:teuthology.orchestra.run.smithi122.stdout:Install 6 Packages 2024-09-11T07:58:16.484 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T07:58:16.489 INFO:teuthology.orchestra.run.smithi122.stdout:Total download size: 2.0 M 2024-09-11T07:58:16.489 INFO:teuthology.orchestra.run.smithi122.stdout:Installed size: 9.2 M 2024-09-11T07:58:16.489 INFO:teuthology.orchestra.run.smithi122.stdout:Downloading Packages: 2024-09-11T07:58:16.505 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2024-09-11T07:58:16.506 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-09-11T07:58:16.506 INFO:teuthology.orchestra.run.smithi135.stdout: Package Architecture Version Repository Size 2024-09-11T07:58:16.506 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-09-11T07:58:16.506 INFO:teuthology.orchestra.run.smithi135.stdout:Installing: 2024-09-11T07:58:16.506 INFO:teuthology.orchestra.run.smithi135.stdout: nvme-cli x86_64 2.9.1-6.el9 baseos 906 k 2024-09-11T07:58:16.506 INFO:teuthology.orchestra.run.smithi135.stdout: nvmetcli noarch 0.7-3.el9 baseos 44 k 2024-09-11T07:58:16.506 INFO:teuthology.orchestra.run.smithi135.stdout:Installing dependencies: 2024-09-11T07:58:16.506 INFO:teuthology.orchestra.run.smithi135.stdout: libnvme x86_64 1.9-3.el9 baseos 106 k 2024-09-11T07:58:16.507 INFO:teuthology.orchestra.run.smithi135.stdout: python3-configshell noarch 1:1.1.30-1.el9 baseos 72 k 2024-09-11T07:58:16.507 INFO:teuthology.orchestra.run.smithi135.stdout: python3-kmod x86_64 0.9-32.el9 baseos 84 k 2024-09-11T07:58:16.507 INFO:teuthology.orchestra.run.smithi135.stdout: python3-urwid x86_64 2.1.2-4.el9 baseos 837 k 2024-09-11T07:58:16.507 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-11T07:58:16.507 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction Summary 2024-09-11T07:58:16.507 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-09-11T07:58:16.507 INFO:teuthology.orchestra.run.smithi135.stdout:Install 6 Packages 2024-09-11T07:58:16.507 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-11T07:58:16.511 INFO:teuthology.orchestra.run.smithi135.stdout:Total download size: 2.0 M 2024-09-11T07:58:16.511 INFO:teuthology.orchestra.run.smithi135.stdout:Installed size: 9.2 M 2024-09-11T07:58:16.511 INFO:teuthology.orchestra.run.smithi135.stdout:Downloading Packages: 2024-09-11T07:58:16.653 INFO:teuthology.orchestra.run.smithi135.stdout:(1/6): nvmetcli-0.7-3.el9.noarch.rpm 502 kB/s | 44 kB 00:00 2024-09-11T07:58:16.659 INFO:teuthology.orchestra.run.smithi122.stdout:(1/6): libnvme-1.9-3.el9.x86_64.rpm 899 kB/s | 106 kB 00:00 2024-09-11T07:58:16.679 INFO:teuthology.orchestra.run.smithi135.stdout:(2/6): libnvme-1.9-3.el9.x86_64.rpm 930 kB/s | 106 kB 00:00 2024-09-11T07:58:16.709 INFO:teuthology.orchestra.run.smithi122.stdout:(2/6): python3-configshell-1.1.30-1.el9.noarch. 1.4 MB/s | 72 kB 00:00 2024-09-11T07:58:16.712 INFO:teuthology.orchestra.run.smithi135.stdout:(3/6): python3-configshell-1.1.30-1.el9.noarch. 1.2 MB/s | 72 kB 00:00 2024-09-11T07:58:16.737 INFO:teuthology.orchestra.run.smithi135.stdout:(4/6): python3-kmod-0.9-32.el9.x86_64.rpm 1.4 MB/s | 84 kB 00:00 2024-09-11T07:58:16.753 INFO:teuthology.orchestra.run.smithi122.stdout:(3/6): nvme-cli-2.9.1-6.el9.x86_64.rpm 4.2 MB/s | 906 kB 00:00 2024-09-11T07:58:16.777 INFO:teuthology.orchestra.run.smithi122.stdout:(4/6): nvmetcli-0.7-3.el9.noarch.rpm 189 kB/s | 44 kB 00:00 2024-09-11T07:58:16.802 INFO:teuthology.orchestra.run.smithi122.stdout:(5/6): python3-kmod-0.9-32.el9.x86_64.rpm 914 kB/s | 84 kB 00:00 2024-09-11T07:58:16.821 INFO:teuthology.orchestra.run.smithi135.stdout:(5/6): nvme-cli-2.9.1-6.el9.x86_64.rpm 3.4 MB/s | 906 kB 00:00 2024-09-11T07:58:16.835 INFO:teuthology.orchestra.run.smithi122.stdout:(6/6): python3-urwid-2.1.2-4.el9.x86_64.rpm 9.9 MB/s | 837 kB 00:00 2024-09-11T07:58:16.836 INFO:teuthology.orchestra.run.smithi122.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:58:16.836 INFO:teuthology.orchestra.run.smithi122.stdout:Total 5.8 MB/s | 2.0 MB 00:00 2024-09-11T07:58:16.871 INFO:teuthology.orchestra.run.smithi135.stdout:(6/6): python3-urwid-2.1.2-4.el9.x86_64.rpm 5.2 MB/s | 837 kB 00:00 2024-09-11T07:58:16.871 INFO:teuthology.orchestra.run.smithi135.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:58:16.872 INFO:teuthology.orchestra.run.smithi135.stdout:Total 5.6 MB/s | 2.0 MB 00:00 2024-09-11T07:58:16.909 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction check 2024-09-11T07:58:16.923 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction check succeeded. 2024-09-11T07:58:16.923 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction test 2024-09-11T07:58:16.945 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction check 2024-09-11T07:58:16.959 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction check succeeded. 2024-09-11T07:58:16.959 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction test 2024-09-11T07:58:17.225 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction test succeeded. 2024-09-11T07:58:17.226 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction 2024-09-11T07:58:17.263 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction test succeeded. 2024-09-11T07:58:17.263 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction 2024-09-11T07:58:17.649 INFO:teuthology.orchestra.run.smithi122.stdout: Preparing : 1/1 2024-09-11T07:58:17.690 INFO:teuthology.orchestra.run.smithi135.stdout: Preparing : 1/1 2024-09-11T07:58:17.747 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-urwid-2.1.2-4.el9.x86_64 1/6 2024-09-11T07:58:17.781 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-configshell-1:1.1.30-1.el9.noarch 2/6 2024-09-11T07:58:17.798 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-urwid-2.1.2-4.el9.x86_64 1/6 2024-09-11T07:58:17.809 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-kmod-0.9-32.el9.x86_64 3/6 2024-09-11T07:58:17.833 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-configshell-1:1.1.30-1.el9.noarch 2/6 2024-09-11T07:58:17.870 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-kmod-0.9-32.el9.x86_64 3/6 2024-09-11T07:58:18.026 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libnvme-1.9-3.el9.x86_64 4/6 2024-09-11T07:58:18.066 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : nvme-cli-2.9.1-6.el9.x86_64 5/6 2024-09-11T07:58:18.076 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libnvme-1.9-3.el9.x86_64 4/6 2024-09-11T07:58:18.118 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : nvme-cli-2.9.1-6.el9.x86_64 5/6 2024-09-11T07:58:18.511 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: nvme-cli-2.9.1-6.el9.x86_64 5/6 2024-09-11T07:58:18.543 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : nvmetcli-0.7-3.el9.noarch 6/6 2024-09-11T07:58:18.552 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: nvme-cli-2.9.1-6.el9.x86_64 5/6 2024-09-11T07:58:18.580 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : nvmetcli-0.7-3.el9.noarch 6/6 2024-09-11T07:58:19.564 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: nvmetcli-0.7-3.el9.noarch 6/6 2024-09-11T07:58:19.565 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/6 2024-09-11T07:58:19.565 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/6 2024-09-11T07:58:19.565 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : nvmetcli-0.7-3.el9.noarch 3/6 2024-09-11T07:58:19.565 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-configshell-1:1.1.30-1.el9.noarch 4/6 2024-09-11T07:58:19.565 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-kmod-0.9-32.el9.x86_64 5/6 2024-09-11T07:58:19.636 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: nvmetcli-0.7-3.el9.noarch 6/6 2024-09-11T07:58:19.636 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/6 2024-09-11T07:58:19.636 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/6 2024-09-11T07:58:19.636 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : nvmetcli-0.7-3.el9.noarch 3/6 2024-09-11T07:58:19.636 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-configshell-1:1.1.30-1.el9.noarch 4/6 2024-09-11T07:58:19.636 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-kmod-0.9-32.el9.x86_64 5/6 2024-09-11T07:58:19.947 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-urwid-2.1.2-4.el9.x86_64 6/6 2024-09-11T07:58:19.947 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T07:58:19.947 INFO:teuthology.orchestra.run.smithi122.stdout:Installed: 2024-09-11T07:58:19.947 INFO:teuthology.orchestra.run.smithi122.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-11T07:58:19.947 INFO:teuthology.orchestra.run.smithi122.stdout: nvmetcli-0.7-3.el9.noarch python3-configshell-1:1.1.30-1.el9.noarch 2024-09-11T07:58:19.947 INFO:teuthology.orchestra.run.smithi122.stdout: python3-kmod-0.9-32.el9.x86_64 python3-urwid-2.1.2-4.el9.x86_64 2024-09-11T07:58:19.947 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T07:58:19.947 INFO:teuthology.orchestra.run.smithi122.stdout:Complete! 2024-09-11T07:58:19.971 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-urwid-2.1.2-4.el9.x86_64 6/6 2024-09-11T07:58:19.971 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-11T07:58:19.972 INFO:teuthology.orchestra.run.smithi135.stdout:Installed: 2024-09-11T07:58:19.972 INFO:teuthology.orchestra.run.smithi135.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-11T07:58:19.972 INFO:teuthology.orchestra.run.smithi135.stdout: nvmetcli-0.7-3.el9.noarch python3-configshell-1:1.1.30-1.el9.noarch 2024-09-11T07:58:19.972 INFO:teuthology.orchestra.run.smithi135.stdout: python3-kmod-0.9-32.el9.x86_64 python3-urwid-2.1.2-4.el9.x86_64 2024-09-11T07:58:19.972 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-11T07:58:19.972 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2024-09-11T07:58:20.248 DEBUG:teuthology.parallel:result is None 2024-09-11T07:58:20.284 DEBUG:teuthology.parallel:result is None 2024-09-11T07:58:20.284 INFO:teuthology.run_tasks:Running task install... 2024-09-11T07:58:20.293 DEBUG:teuthology.task.install:project ceph 2024-09-11T07:58:20.294 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd0e6828a2016d48cf25ad84064e50742bb1c39b9'}, 'extra_packages': ['cephadm']} 2024-09-11T07:58:20.294 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': 'd0e6828a2016d48cf25ad84064e50742bb1c39b9'} 2024-09-11T07:58:20.294 INFO:teuthology.task.install:Using flavor: default 2024-09-11T07:58:20.299 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-11T07:58:20.299 INFO:teuthology.task.install:extra packages: [] 2024-09-11T07:58:20.299 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd0e6828a2016d48cf25ad84064e50742bb1c39b9', 'tag': None, 'wait_for_package': False} 2024-09-11T07:58:20.300 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d0e6828a2016d48cf25ad84064e50742bb1c39b9 2024-09-11T07:58:20.302 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd0e6828a2016d48cf25ad84064e50742bb1c39b9', 'tag': None, 'wait_for_package': False} 2024-09-11T07:58:20.302 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d0e6828a2016d48cf25ad84064e50742bb1c39b9 2024-09-11T07:58:20.449 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-10-2008-quincy/d0e6828a2016d48cf25ad84064e50742bb1c39b9/centos/9/flavors/default/ 2024-09-11T07:58:20.449 INFO:teuthology.task.install.rpm:Package version is 17.2.7-1623.gd0e6828a 2024-09-11T07:58:20.456 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-10-2008-quincy/d0e6828a2016d48cf25ad84064e50742bb1c39b9/centos/9/flavors/default/ 2024-09-11T07:58:20.457 INFO:teuthology.task.install.rpm:Package version is 17.2.7-1623.gd0e6828a 2024-09-11T07:58:20.557 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-10-2008-quincy/d0e6828a2016d48cf25ad84064e50742bb1c39b9/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-10-2008-quincy/d0e6828a2016d48cf25ad84064e50742bb1c39b9/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-10-2008-quincy/d0e6828a2016d48cf25ad84064e50742bb1c39b9/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-11T07:58:20.558 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-09-11T07:58:20.558 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-11T07:58:20.566 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-10-2008-quincy/d0e6828a2016d48cf25ad84064e50742bb1c39b9/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-10-2008-quincy/d0e6828a2016d48cf25ad84064e50742bb1c39b9/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-10-2008-quincy/d0e6828a2016d48cf25ad84064e50742bb1c39b9/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-11T07:58:20.566 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-11T07:58:20.566 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-11T07:58:20.587 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-11T07:58:20.587 DEBUG:teuthology.orchestra.run.smithi135:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d0e6828a2016d48cf25ad84064e50742bb1c39b9/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-11T07:58:20.594 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-11T07:58:20.595 DEBUG:teuthology.orchestra.run.smithi122:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d0e6828a2016d48cf25ad84064e50742bb1c39b9/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-11T07:58:20.653 DEBUG:teuthology.orchestra.run.smithi135:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-11T07:58:20.660 DEBUG:teuthology.orchestra.run.smithi122:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-11T07:58:20.728 DEBUG:teuthology.orchestra.run.smithi135:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-11T07:58:20.735 DEBUG:teuthology.orchestra.run.smithi122:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-11T07:58:20.793 INFO:teuthology.orchestra.run.smithi135.stdout:check_obsoletes = 1 2024-09-11T07:58:20.794 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum clean all 2024-09-11T07:58:20.799 INFO:teuthology.orchestra.run.smithi122.stdout:check_obsoletes = 1 2024-09-11T07:58:20.802 DEBUG:teuthology.orchestra.run.smithi122:> sudo yum clean all 2024-09-11T07:58:21.031 INFO:teuthology.orchestra.run.smithi135.stdout:45 files removed 2024-09-11T07:58:21.031 INFO:teuthology.orchestra.run.smithi122.stdout:45 files removed 2024-09-11T07:58:21.058 DEBUG:teuthology.orchestra.run.smithi122:> sudo yum -y install ceph-radosgw 2024-09-11T07:58:21.060 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install ceph-radosgw 2024-09-11T07:58:21.609 INFO:teuthology.orchestra.run.smithi122.stdout:ceph packages for x86_64 290 kB/s | 77 kB 00:00 2024-09-11T07:58:21.619 INFO:teuthology.orchestra.run.smithi135.stdout:ceph packages for x86_64 287 kB/s | 77 kB 00:00 2024-09-11T07:58:21.886 INFO:teuthology.orchestra.run.smithi135.stdout:ceph noarch packages 48 kB/s | 11 kB 00:00 2024-09-11T07:58:21.902 INFO:teuthology.orchestra.run.smithi122.stdout:ceph noarch packages 43 kB/s | 11 kB 00:00 2024-09-11T07:58:22.145 INFO:teuthology.orchestra.run.smithi135.stdout:ceph source packages 8.0 kB/s | 1.8 kB 00:00 2024-09-11T07:58:22.177 INFO:teuthology.orchestra.run.smithi122.stdout:ceph source packages 7.4 kB/s | 1.8 kB 00:00 2024-09-11T07:58:22.687 INFO:teuthology.orchestra.run.smithi135.stdout:CentOS Stream 9 - BaseOS 16 MB/s | 8.2 MB 00:00 2024-09-11T07:58:22.743 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS Stream 9 - BaseOS 16 MB/s | 8.2 MB 00:00 2024-09-11T07:58:24.737 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS Stream 9 - AppStream 22 MB/s | 20 MB 00:00 2024-09-11T07:58:26.055 INFO:teuthology.orchestra.run.smithi135.stdout:CentOS Stream 9 - AppStream 9.1 MB/s | 20 MB 00:02 2024-09-11T07:58:29.259 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS Stream 9 - CRB 8.9 MB/s | 6.5 MB 00:00 2024-09-11T07:58:30.488 INFO:teuthology.orchestra.run.smithi135.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.5 MB 00:00 2024-09-11T07:58:30.871 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS Stream 9 - Extras packages 61 kB/s | 19 kB 00:00 2024-09-11T07:58:31.817 INFO:teuthology.orchestra.run.smithi122.stdout:Extra Packages for Enterprise Linux 28 MB/s | 23 MB 00:00 2024-09-11T07:58:32.148 INFO:teuthology.orchestra.run.smithi135.stdout:CentOS Stream 9 - Extras packages 61 kB/s | 19 kB 00:00 2024-09-11T07:58:33.085 INFO:teuthology.orchestra.run.smithi135.stdout:Extra Packages for Enterprise Linux 28 MB/s | 23 MB 00:00 2024-09-11T07:58:37.198 INFO:teuthology.orchestra.run.smithi122.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-11T07:58:38.633 INFO:teuthology.orchestra.run.smithi135.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-09-11T07:58:39.582 INFO:teuthology.orchestra.run.smithi122.stdout:Dependencies resolved. 2024-09-11T07:58:39.584 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-11T07:58:39.584 INFO:teuthology.orchestra.run.smithi122.stdout: Package Arch Version Repository Size 2024-09-11T07:58:39.584 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-11T07:58:39.584 INFO:teuthology.orchestra.run.smithi122.stdout:Installing: 2024-09-11T07:58:39.584 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-radosgw x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 14 M 2024-09-11T07:58:39.584 INFO:teuthology.orchestra.run.smithi122.stdout:Upgrading: 2024-09-11T07:58:39.584 INFO:teuthology.orchestra.run.smithi122.stdout: librados2 x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 3.1 M 2024-09-11T07:58:39.584 INFO:teuthology.orchestra.run.smithi122.stdout: librbd1 x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 2.9 M 2024-09-11T07:58:39.584 INFO:teuthology.orchestra.run.smithi122.stdout:Installing dependencies: 2024-09-11T07:58:39.584 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-base x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 5.0 M 2024-09-11T07:58:39.585 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-common x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 23 M 2024-09-11T07:58:39.585 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-selinux x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 25 k 2024-09-11T07:58:39.585 INFO:teuthology.orchestra.run.smithi122.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-11T07:58:39.585 INFO:teuthology.orchestra.run.smithi122.stdout: libcephfs2 x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 636 k 2024-09-11T07:58:39.585 INFO:teuthology.orchestra.run.smithi122.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-11T07:58:39.585 INFO:teuthology.orchestra.run.smithi122.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-11T07:58:39.585 INFO:teuthology.orchestra.run.smithi122.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-11T07:58:39.585 INFO:teuthology.orchestra.run.smithi122.stdout: libradosstriper1 x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 457 k 2024-09-11T07:58:39.585 INFO:teuthology.orchestra.run.smithi122.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-11T07:58:39.585 INFO:teuthology.orchestra.run.smithi122.stdout: librgw2 x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 4.9 M 2024-09-11T07:58:39.585 INFO:teuthology.orchestra.run.smithi122.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-11T07:58:39.586 INFO:teuthology.orchestra.run.smithi122.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-11T07:58:39.586 INFO:teuthology.orchestra.run.smithi122.stdout: python3-ceph-argparse x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 45 k 2024-09-11T07:58:39.586 INFO:teuthology.orchestra.run.smithi122.stdout: python3-ceph-common x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 120 k 2024-09-11T07:58:39.586 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cephfs x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 156 k 2024-09-11T07:58:39.586 INFO:teuthology.orchestra.run.smithi122.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-11T07:58:39.586 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rados x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 320 k 2024-09-11T07:58:39.586 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rbd x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 294 k 2024-09-11T07:58:39.586 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rgw x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 99 k 2024-09-11T07:58:39.586 INFO:teuthology.orchestra.run.smithi122.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-11T07:58:39.586 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T07:58:39.586 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction Summary 2024-09-11T07:58:39.586 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-11T07:58:39.587 INFO:teuthology.orchestra.run.smithi122.stdout:Install 22 Packages 2024-09-11T07:58:39.587 INFO:teuthology.orchestra.run.smithi122.stdout:Upgrade 2 Packages 2024-09-11T07:58:39.587 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T07:58:39.587 INFO:teuthology.orchestra.run.smithi122.stdout:Total download size: 58 M 2024-09-11T07:58:39.587 INFO:teuthology.orchestra.run.smithi122.stdout:Downloading Packages: 2024-09-11T07:58:40.525 INFO:teuthology.orchestra.run.smithi122.stdout:(1/24): ceph-base-17.2.7-1623.gd0e6828a.el9.x86 9.7 MB/s | 5.0 MB 00:00 2024-09-11T07:58:40.591 INFO:teuthology.orchestra.run.smithi122.stdout:(2/24): ceph-selinux-17.2.7-1623.gd0e6828a.el9. 378 kB/s | 25 kB 00:00 2024-09-11T07:58:40.700 INFO:teuthology.orchestra.run.smithi122.stdout:(3/24): libcephfs2-17.2.7-1623.gd0e6828a.el9.x8 5.7 MB/s | 636 kB 00:00 2024-09-11T07:58:40.925 INFO:teuthology.orchestra.run.smithi122.stdout:(4/24): ceph-radosgw-17.2.7-1623.gd0e6828a.el9. 16 MB/s | 14 MB 00:00 2024-09-11T07:58:40.951 INFO:teuthology.orchestra.run.smithi122.stdout:(5/24): libradosstriper1-17.2.7-1623.gd0e6828a. 1.8 MB/s | 457 kB 00:00 2024-09-11T07:58:41.009 INFO:teuthology.orchestra.run.smithi122.stdout:(6/24): python3-ceph-argparse-17.2.7-1623.gd0e6 774 kB/s | 45 kB 00:00 2024-09-11T07:58:41.035 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2024-09-11T07:58:41.037 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-09-11T07:58:41.037 INFO:teuthology.orchestra.run.smithi135.stdout: Package Arch Version Repository Size 2024-09-11T07:58:41.037 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-09-11T07:58:41.037 INFO:teuthology.orchestra.run.smithi135.stdout:Installing: 2024-09-11T07:58:41.038 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-radosgw x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 14 M 2024-09-11T07:58:41.038 INFO:teuthology.orchestra.run.smithi135.stdout:Upgrading: 2024-09-11T07:58:41.038 INFO:teuthology.orchestra.run.smithi135.stdout: librados2 x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 3.1 M 2024-09-11T07:58:41.038 INFO:teuthology.orchestra.run.smithi135.stdout: librbd1 x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 2.9 M 2024-09-11T07:58:41.039 INFO:teuthology.orchestra.run.smithi135.stdout:Installing dependencies: 2024-09-11T07:58:41.039 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-base x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 5.0 M 2024-09-11T07:58:41.039 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-common x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 23 M 2024-09-11T07:58:41.039 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-selinux x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 25 k 2024-09-11T07:58:41.039 INFO:teuthology.orchestra.run.smithi135.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-11T07:58:41.039 INFO:teuthology.orchestra.run.smithi135.stdout: libcephfs2 x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 636 k 2024-09-11T07:58:41.040 INFO:teuthology.orchestra.run.smithi135.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-11T07:58:41.040 INFO:teuthology.orchestra.run.smithi135.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-11T07:58:41.040 INFO:teuthology.orchestra.run.smithi135.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-11T07:58:41.040 INFO:teuthology.orchestra.run.smithi135.stdout: libradosstriper1 x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 457 k 2024-09-11T07:58:41.040 INFO:teuthology.orchestra.run.smithi135.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-11T07:58:41.041 INFO:teuthology.orchestra.run.smithi135.stdout: librgw2 x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 4.9 M 2024-09-11T07:58:41.041 INFO:teuthology.orchestra.run.smithi135.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-11T07:58:41.041 INFO:teuthology.orchestra.run.smithi135.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-11T07:58:41.041 INFO:teuthology.orchestra.run.smithi135.stdout: python3-ceph-argparse x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 45 k 2024-09-11T07:58:41.041 INFO:teuthology.orchestra.run.smithi135.stdout: python3-ceph-common x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 120 k 2024-09-11T07:58:41.042 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cephfs x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 156 k 2024-09-11T07:58:41.042 INFO:teuthology.orchestra.run.smithi135.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-11T07:58:41.042 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rados x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 320 k 2024-09-11T07:58:41.042 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rbd x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 294 k 2024-09-11T07:58:41.042 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rgw x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 99 k 2024-09-11T07:58:41.043 INFO:teuthology.orchestra.run.smithi135.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-11T07:58:41.043 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-11T07:58:41.043 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction Summary 2024-09-11T07:58:41.043 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-09-11T07:58:41.043 INFO:teuthology.orchestra.run.smithi135.stdout:Install 22 Packages 2024-09-11T07:58:41.043 INFO:teuthology.orchestra.run.smithi135.stdout:Upgrade 2 Packages 2024-09-11T07:58:41.044 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-11T07:58:41.044 INFO:teuthology.orchestra.run.smithi135.stdout:Total download size: 58 M 2024-09-11T07:58:41.045 INFO:teuthology.orchestra.run.smithi135.stdout:Downloading Packages: 2024-09-11T07:58:41.135 INFO:teuthology.orchestra.run.smithi122.stdout:(7/24): librgw2-17.2.7-1623.gd0e6828a.el9.x86_6 24 MB/s | 4.9 MB 00:00 2024-09-11T07:58:41.160 INFO:teuthology.orchestra.run.smithi122.stdout:(8/24): python3-ceph-common-17.2.7-1623.gd0e682 798 kB/s | 120 kB 00:00 2024-09-11T07:58:41.186 INFO:teuthology.orchestra.run.smithi122.stdout:(9/24): python3-cephfs-17.2.7-1623.gd0e6828a.el 3.0 MB/s | 156 kB 00:00 2024-09-11T07:58:41.211 INFO:teuthology.orchestra.run.smithi122.stdout:(10/24): python3-rados-17.2.7-1623.gd0e6828a.el 6.2 MB/s | 320 kB 00:00 2024-09-11T07:58:41.245 INFO:teuthology.orchestra.run.smithi122.stdout:(11/24): python3-rbd-17.2.7-1623.gd0e6828a.el9. 4.9 MB/s | 294 kB 00:00 2024-09-11T07:58:41.270 INFO:teuthology.orchestra.run.smithi122.stdout:(12/24): python3-rgw-17.2.7-1623.gd0e6828a.el9. 1.7 MB/s | 99 kB 00:00 2024-09-11T07:58:41.495 INFO:teuthology.orchestra.run.smithi122.stdout:(13/24): ceph-common-17.2.7-1623.gd0e6828a.el9. 15 MB/s | 23 MB 00:01 2024-09-11T07:58:41.787 INFO:teuthology.orchestra.run.smithi122.stdout:(14/24): librabbitmq-0.11.0-7.el9.x86_64.rpm 87 kB/s | 45 kB 00:00 2024-09-11T07:58:41.962 INFO:teuthology.orchestra.run.smithi122.stdout:(15/24): libpmemobj-1.12.1-1.el9.x86_64.rpm 223 kB/s | 160 kB 00:00 2024-09-11T07:58:42.079 INFO:teuthology.orchestra.run.smithi122.stdout:(16/24): python3-prettytable-0.7.2-27.el9.noarc 362 kB/s | 42 kB 00:00 2024-09-11T07:58:42.130 INFO:teuthology.orchestra.run.smithi122.stdout:(17/24): gperftools-libs-2.9.1-2.el9.x86_64.rpm 6.0 MB/s | 309 kB 00:00 2024-09-11T07:58:42.155 INFO:teuthology.orchestra.run.smithi122.stdout:(18/24): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-11T07:58:42.188 INFO:teuthology.orchestra.run.smithi122.stdout:(19/24): lttng-ust-2.12.0-6.el9.x86_64.rpm 729 kB/s | 292 kB 00:00 2024-09-11T07:58:42.214 INFO:teuthology.orchestra.run.smithi122.stdout:(20/24): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-09-11T07:58:42.247 INFO:teuthology.orchestra.run.smithi122.stdout:(21/24): librdkafka-1.6.1-102.el9.x86_64.rpm 881 kB/s | 662 kB 00:00 2024-09-11T07:58:42.306 INFO:teuthology.orchestra.run.smithi122.stdout:(22/24): thrift-0.15.0-2.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-09-11T07:58:42.442 INFO:teuthology.orchestra.run.smithi135.stdout:(1/24): ceph-base-17.2.7-1623.gd0e6828a.el9.x86 7.1 MB/s | 5.0 MB 00:00 2024-09-11T07:58:42.473 INFO:teuthology.orchestra.run.smithi122.stdout:(23/24): librados2-17.2.7-1623.gd0e6828a.el9.x8 12 MB/s | 3.1 MB 00:00 2024-09-11T07:58:42.484 INFO:teuthology.orchestra.run.smithi135.stdout:(2/24): ceph-selinux-17.2.7-1623.gd0e6828a.el9. 609 kB/s | 25 kB 00:00 2024-09-11T07:58:42.523 INFO:teuthology.orchestra.run.smithi122.stdout:(24/24): librbd1-17.2.7-1623.gd0e6828a.el9.x86_ 11 MB/s | 2.9 MB 00:00 2024-09-11T07:58:42.527 INFO:teuthology.orchestra.run.smithi122.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:58:42.528 INFO:teuthology.orchestra.run.smithi122.stdout:Total 20 MB/s | 58 MB 00:02 2024-09-11T07:58:42.584 INFO:teuthology.orchestra.run.smithi135.stdout:(3/24): libcephfs2-17.2.7-1623.gd0e6828a.el9.x8 6.2 MB/s | 636 kB 00:00 2024-09-11T07:58:42.589 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction check 2024-09-11T07:58:42.622 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction check succeeded. 2024-09-11T07:58:42.623 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction test 2024-09-11T07:58:42.735 INFO:teuthology.orchestra.run.smithi135.stdout:(4/24): ceph-radosgw-17.2.7-1623.gd0e6828a.el9. 15 MB/s | 14 MB 00:00 2024-09-11T07:58:42.776 INFO:teuthology.orchestra.run.smithi135.stdout:(5/24): libradosstriper1-17.2.7-1623.gd0e6828a. 2.3 MB/s | 457 kB 00:00 2024-09-11T07:58:42.844 INFO:teuthology.orchestra.run.smithi135.stdout:(6/24): python3-ceph-argparse-17.2.7-1623.gd0e6 678 kB/s | 45 kB 00:00 2024-09-11T07:58:42.935 INFO:teuthology.orchestra.run.smithi135.stdout:(7/24): librgw2-17.2.7-1623.gd0e6828a.el9.x86_6 25 MB/s | 4.9 MB 00:00 2024-09-11T07:58:42.960 INFO:teuthology.orchestra.run.smithi135.stdout:(8/24): python3-ceph-common-17.2.7-1623.gd0e682 1.0 MB/s | 120 kB 00:00 2024-09-11T07:58:42.986 INFO:teuthology.orchestra.run.smithi135.stdout:(9/24): python3-cephfs-17.2.7-1623.gd0e6828a.el 3.0 MB/s | 156 kB 00:00 2024-09-11T07:58:43.011 INFO:teuthology.orchestra.run.smithi135.stdout:(10/24): python3-rados-17.2.7-1623.gd0e6828a.el 6.2 MB/s | 320 kB 00:00 2024-09-11T07:58:43.036 INFO:teuthology.orchestra.run.smithi135.stdout:(11/24): python3-rbd-17.2.7-1623.gd0e6828a.el9. 5.8 MB/s | 294 kB 00:00 2024-09-11T07:58:43.062 INFO:teuthology.orchestra.run.smithi135.stdout:(12/24): python3-rgw-17.2.7-1623.gd0e6828a.el9. 1.9 MB/s | 99 kB 00:00 2024-09-11T07:58:43.080 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction test succeeded. 2024-09-11T07:58:43.080 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction 2024-09-11T07:58:43.337 INFO:teuthology.orchestra.run.smithi135.stdout:(13/24): ceph-common-17.2.7-1623.gd0e6828a.el9. 14 MB/s | 23 MB 00:01 2024-09-11T07:58:43.612 INFO:teuthology.orchestra.run.smithi135.stdout:(14/24): librabbitmq-0.11.0-7.el9.x86_64.rpm 82 kB/s | 45 kB 00:00 2024-09-11T07:58:43.796 INFO:teuthology.orchestra.run.smithi135.stdout:(15/24): libpmemobj-1.12.1-1.el9.x86_64.rpm 211 kB/s | 160 kB 00:00 2024-09-11T07:58:43.905 INFO:teuthology.orchestra.run.smithi135.stdout:(16/24): python3-prettytable-0.7.2-27.el9.noarc 390 kB/s | 42 kB 00:00 2024-09-11T07:58:43.955 INFO:teuthology.orchestra.run.smithi135.stdout:(17/24): gperftools-libs-2.9.1-2.el9.x86_64.rpm 6.1 MB/s | 309 kB 00:00 2024-09-11T07:58:43.972 INFO:teuthology.orchestra.run.smithi135.stdout:(18/24): liboath-2.6.7-2.el9.x86_64.rpm 2.9 MB/s | 49 kB 00:00 2024-09-11T07:58:44.005 INFO:teuthology.orchestra.run.smithi135.stdout:(19/24): lttng-ust-2.12.0-6.el9.x86_64.rpm 745 kB/s | 292 kB 00:00 2024-09-11T07:58:44.031 INFO:teuthology.orchestra.run.smithi135.stdout:(20/24): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-09-11T07:58:44.114 INFO:teuthology.orchestra.run.smithi135.stdout:(21/24): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-09-11T07:58:44.215 INFO:teuthology.orchestra.run.smithi135.stdout:(22/24): librdkafka-1.6.1-102.el9.x86_64.rpm 755 kB/s | 662 kB 00:00 2024-09-11T07:58:44.262 INFO:teuthology.orchestra.run.smithi122.stdout: Preparing : 1/1 2024-09-11T07:58:44.282 INFO:teuthology.orchestra.run.smithi135.stdout:(23/24): librados2-17.2.7-1623.gd0e6828a.el9.x8 13 MB/s | 3.1 MB 00:00 2024-09-11T07:58:44.357 INFO:teuthology.orchestra.run.smithi135.stdout:(24/24): librbd1-17.2.7-1623.gd0e6828a.el9.x86_ 12 MB/s | 2.9 MB 00:00 2024-09-11T07:58:44.362 INFO:teuthology.orchestra.run.smithi135.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:58:44.362 INFO:teuthology.orchestra.run.smithi135.stdout:Total 17 MB/s | 58 MB 00:03 2024-09-11T07:58:44.426 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction check 2024-09-11T07:58:44.458 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction check succeeded. 2024-09-11T07:58:44.458 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction test 2024-09-11T07:58:44.607 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/26 2024-09-11T07:58:44.717 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : librados2-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/26 2024-09-11T07:58:44.812 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: librados2-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/26 2024-09-11T07:58:44.894 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-rados-2:17.2.7-1623.gd0e6828a.el9.x86_64 3/26 2024-09-11T07:58:44.927 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libcephfs2-2:17.2.7-1623.gd0e6828a.el9.x86_64 4/26 2024-09-11T07:58:44.932 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction test succeeded. 2024-09-11T07:58:44.933 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction 2024-09-11T07:58:45.015 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: libcephfs2-2:17.2.7-1623.gd0e6828a.el9.x86_64 4/26 2024-09-11T07:58:45.061 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : thrift-0.15.0-2.el9.x86_64 5/26 2024-09-11T07:58:45.090 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/26 2024-09-11T07:58:45.574 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/26 2024-09-11T07:58:45.605 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : librgw2-2:17.2.7-1623.gd0e6828a.el9.x86_64 8/26 2024-09-11T07:58:45.668 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: librgw2-2:17.2.7-1623.gd0e6828a.el9.x86_64 8/26 2024-09-11T07:58:45.705 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/26 2024-09-11T07:58:45.747 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-ceph-argparse-2:17.2.7-1623.gd0e6828a.el9. 10/26 2024-09-11T07:58:45.784 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-cephfs-2:17.2.7-1623.gd0e6828a.el9.x86_64 11/26 2024-09-11T07:58:45.846 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-rgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 12/26 2024-09-11T07:58:45.874 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libradosstriper1-2:17.2.7-1623.gd0e6828a.el9.x86_6 13/26 2024-09-11T07:58:45.940 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: libradosstriper1-2:17.2.7-1623.gd0e6828a.el9.x86_6 13/26 2024-09-11T07:58:45.980 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/26 2024-09-11T07:58:46.016 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/26 2024-09-11T07:58:46.050 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 16/26 2024-09-11T07:58:46.105 INFO:teuthology.orchestra.run.smithi135.stdout: Preparing : 1/1 2024-09-11T07:58:46.344 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/26 2024-09-11T07:58:46.389 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : librbd1-2:17.2.7-1623.gd0e6828a.el9.x86_64 18/26 2024-09-11T07:58:46.479 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: librbd1-2:17.2.7-1623.gd0e6828a.el9.x86_64 18/26 2024-09-11T07:58:46.506 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/26 2024-09-11T07:58:46.552 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-rbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 19/26 2024-09-11T07:58:46.576 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : librados2-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/26 2024-09-11T07:58:46.591 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-ceph-common-2:17.2.7-1623.gd0e6828a.el9.x8 20/26 2024-09-11T07:58:46.652 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: librados2-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/26 2024-09-11T07:58:46.738 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-rados-2:17.2.7-1623.gd0e6828a.el9.x86_64 3/26 2024-09-11T07:58:46.769 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libcephfs2-2:17.2.7-1623.gd0e6828a.el9.x86_64 4/26 2024-09-11T07:58:46.857 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: libcephfs2-2:17.2.7-1623.gd0e6828a.el9.x86_64 4/26 2024-09-11T07:58:46.915 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : thrift-0.15.0-2.el9.x86_64 5/26 2024-09-11T07:58:46.949 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/26 2024-09-11T07:58:47.398 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/26 2024-09-11T07:58:47.422 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : librgw2-2:17.2.7-1623.gd0e6828a.el9.x86_64 8/26 2024-09-11T07:58:47.493 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: librgw2-2:17.2.7-1623.gd0e6828a.el9.x86_64 8/26 2024-09-11T07:58:47.530 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/26 2024-09-11T07:58:47.582 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-ceph-argparse-2:17.2.7-1623.gd0e6828a.el9. 10/26 2024-09-11T07:58:47.618 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-cephfs-2:17.2.7-1623.gd0e6828a.el9.x86_64 11/26 2024-09-11T07:58:47.681 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-rgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 12/26 2024-09-11T07:58:47.716 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libradosstriper1-2:17.2.7-1623.gd0e6828a.el9.x86_6 13/26 2024-09-11T07:58:47.782 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: libradosstriper1-2:17.2.7-1623.gd0e6828a.el9.x86_6 13/26 2024-09-11T07:58:47.841 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/26 2024-09-11T07:58:47.883 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/26 2024-09-11T07:58:47.916 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 16/26 2024-09-11T07:58:48.211 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/26 2024-09-11T07:58:48.253 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : librbd1-2:17.2.7-1623.gd0e6828a.el9.x86_64 18/26 2024-09-11T07:58:48.336 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: librbd1-2:17.2.7-1623.gd0e6828a.el9.x86_64 18/26 2024-09-11T07:58:48.408 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-rbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 19/26 2024-09-11T07:58:48.439 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-ceph-common-2:17.2.7-1623.gd0e6828a.el9.x8 20/26 2024-09-11T07:58:49.042 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-common-2:17.2.7-1623.gd0e6828a.el9.x86_64 21/26 2024-09-11T07:58:49.080 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-common-2:17.2.7-1623.gd0e6828a.el9.x86_64 21/26 2024-09-11T07:58:49.590 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-common-2:17.2.7-1623.gd0e6828a.el9.x86_64 21/26 2024-09-11T07:58:49.633 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-base-2:17.2.7-1623.gd0e6828a.el9.x86_64 22/26 2024-09-11T07:58:49.732 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-base-2:17.2.7-1623.gd0e6828a.el9.x86_64 22/26 2024-09-11T07:58:49.732 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-11T07:58:49.732 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-11T07:58:49.732 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T07:58:49.767 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-selinux-2:17.2.7-1623.gd0e6828a.el9.x86_64 23/26 2024-09-11T07:58:50.878 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-common-2:17.2.7-1623.gd0e6828a.el9.x86_64 21/26 2024-09-11T07:58:50.928 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-common-2:17.2.7-1623.gd0e6828a.el9.x86_64 21/26 2024-09-11T07:58:51.449 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-common-2:17.2.7-1623.gd0e6828a.el9.x86_64 21/26 2024-09-11T07:58:51.489 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-base-2:17.2.7-1623.gd0e6828a.el9.x86_64 22/26 2024-09-11T07:58:51.582 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-base-2:17.2.7-1623.gd0e6828a.el9.x86_64 22/26 2024-09-11T07:58:51.582 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-11T07:58:51.582 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-11T07:58:51.582 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-11T07:58:51.624 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-selinux-2:17.2.7-1623.gd0e6828a.el9.x86_64 23/26 2024-09-11T07:58:59.505 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-selinux-2:17.2.7-1623.gd0e6828a.el9.x86_64 23/26 2024-09-11T07:58:59.506 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /sys 2024-09-11T07:58:59.506 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /proc 2024-09-11T07:58:59.506 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /mnt 2024-09-11T07:58:59.506 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /var/tmp 2024-09-11T07:58:59.506 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /home 2024-09-11T07:58:59.506 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /root 2024-09-11T07:58:59.506 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /tmp 2024-09-11T07:58:59.506 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T07:59:00.860 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-radosgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 24/26 2024-09-11T07:59:00.938 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 24/26 2024-09-11T07:59:00.939 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T07:59:00.939 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-11T07:59:00.939 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T07:59:00.939 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T07:59:00.939 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T07:59:00.940 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-09-11T07:59:01.022 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-09-11T07:59:01.022 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 26/26 2024-09-11T07:59:01.886 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-selinux-2:17.2.7-1623.gd0e6828a.el9.x86_64 23/26 2024-09-11T07:59:01.887 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /sys 2024-09-11T07:59:01.887 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /proc 2024-09-11T07:59:01.887 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /mnt 2024-09-11T07:59:01.887 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /var/tmp 2024-09-11T07:59:01.887 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /home 2024-09-11T07:59:01.887 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /root 2024-09-11T07:59:01.887 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /tmp 2024-09-11T07:59:01.887 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-11T07:59:02.243 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 26/26 2024-09-11T07:59:02.243 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-base-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/26 2024-09-11T07:59:02.243 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-common-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/26 2024-09-11T07:59:02.243 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-radosgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 3/26 2024-09-11T07:59:02.244 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-selinux-2:17.2.7-1623.gd0e6828a.el9.x86_64 4/26 2024-09-11T07:59:02.244 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libcephfs2-2:17.2.7-1623.gd0e6828a.el9.x86_64 5/26 2024-09-11T07:59:02.244 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libradosstriper1-2:17.2.7-1623.gd0e6828a.el9.x86_6 6/26 2024-09-11T07:59:02.244 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librgw2-2:17.2.7-1623.gd0e6828a.el9.x86_64 7/26 2024-09-11T07:59:02.244 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-ceph-argparse-2:17.2.7-1623.gd0e6828a.el9. 8/26 2024-09-11T07:59:02.244 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-ceph-common-2:17.2.7-1623.gd0e6828a.el9.x8 9/26 2024-09-11T07:59:02.244 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-cephfs-2:17.2.7-1623.gd0e6828a.el9.x86_64 10/26 2024-09-11T07:59:02.244 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-rados-2:17.2.7-1623.gd0e6828a.el9.x86_64 11/26 2024-09-11T07:59:02.244 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-rbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 12/26 2024-09-11T07:59:02.244 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-rgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 13/26 2024-09-11T07:59:02.244 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 14/26 2024-09-11T07:59:02.245 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 15/26 2024-09-11T07:59:02.245 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/26 2024-09-11T07:59:02.245 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/26 2024-09-11T07:59:02.245 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/26 2024-09-11T07:59:02.245 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 19/26 2024-09-11T07:59:02.245 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 20/26 2024-09-11T07:59:02.245 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 21/26 2024-09-11T07:59:02.245 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 22/26 2024-09-11T07:59:02.245 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librados2-2:17.2.7-1623.gd0e6828a.el9.x86_64 23/26 2024-09-11T07:59:02.245 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 24/26 2024-09-11T07:59:02.245 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librbd1-2:17.2.7-1623.gd0e6828a.el9.x86_64 25/26 2024-09-11T07:59:03.144 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 26/26 2024-09-11T07:59:03.144 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T07:59:03.144 INFO:teuthology.orchestra.run.smithi122.stdout:Upgraded: 2024-09-11T07:59:03.144 INFO:teuthology.orchestra.run.smithi122.stdout: librados2-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:59:03.144 INFO:teuthology.orchestra.run.smithi122.stdout: librbd1-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:59:03.144 INFO:teuthology.orchestra.run.smithi122.stdout:Installed: 2024-09-11T07:59:03.144 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-base-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:59:03.144 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-common-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:59:03.144 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-radosgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:59:03.145 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-selinux-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:59:03.145 INFO:teuthology.orchestra.run.smithi122.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-11T07:59:03.145 INFO:teuthology.orchestra.run.smithi122.stdout: libcephfs2-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:59:03.145 INFO:teuthology.orchestra.run.smithi122.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-11T07:59:03.145 INFO:teuthology.orchestra.run.smithi122.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-11T07:59:03.145 INFO:teuthology.orchestra.run.smithi122.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-11T07:59:03.145 INFO:teuthology.orchestra.run.smithi122.stdout: libradosstriper1-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:59:03.145 INFO:teuthology.orchestra.run.smithi122.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-11T07:59:03.145 INFO:teuthology.orchestra.run.smithi122.stdout: librgw2-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:59:03.145 INFO:teuthology.orchestra.run.smithi122.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-11T07:59:03.145 INFO:teuthology.orchestra.run.smithi122.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-11T07:59:03.145 INFO:teuthology.orchestra.run.smithi122.stdout: python3-ceph-argparse-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:59:03.145 INFO:teuthology.orchestra.run.smithi122.stdout: python3-ceph-common-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:59:03.145 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cephfs-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:59:03.146 INFO:teuthology.orchestra.run.smithi122.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-11T07:59:03.146 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rados-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:59:03.146 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:59:03.146 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:59:03.146 INFO:teuthology.orchestra.run.smithi122.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-11T07:59:03.146 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T07:59:03.146 INFO:teuthology.orchestra.run.smithi122.stdout:Complete! 2024-09-11T07:59:03.216 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-radosgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 24/26 2024-09-11T07:59:03.278 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 24/26 2024-09-11T07:59:03.278 INFO:teuthology.orchestra.run.smithi135.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T07:59:03.278 INFO:teuthology.orchestra.run.smithi135.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-11T07:59:03.278 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T07:59:03.278 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T07:59:03.279 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-11T07:59:03.280 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-09-11T07:59:03.369 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-09-11T07:59:03.369 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 26/26 2024-09-11T07:59:03.483 DEBUG:teuthology.orchestra.run.smithi122:> sudo yum -y install ceph-test 2024-09-11T07:59:04.046 INFO:teuthology.orchestra.run.smithi122.stdout:Last metadata expiration check: 0:00:27 ago on Wed 11 Sep 2024 07:58:37 AM UTC. 2024-09-11T07:59:04.178 INFO:teuthology.orchestra.run.smithi122.stdout:Dependencies resolved. 2024-09-11T07:59:04.178 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-11T07:59:04.178 INFO:teuthology.orchestra.run.smithi122.stdout: Package Arch Version Repository Size 2024-09-11T07:59:04.178 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-11T07:59:04.178 INFO:teuthology.orchestra.run.smithi122.stdout:Installing: 2024-09-11T07:59:04.179 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-test x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 48 M 2024-09-11T07:59:04.179 INFO:teuthology.orchestra.run.smithi122.stdout:Installing dependencies: 2024-09-11T07:59:04.179 INFO:teuthology.orchestra.run.smithi122.stdout: libcephsqlite x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 163 k 2024-09-11T07:59:04.179 INFO:teuthology.orchestra.run.smithi122.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-11T07:59:04.179 INFO:teuthology.orchestra.run.smithi122.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-11T07:59:04.179 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T07:59:04.179 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction Summary 2024-09-11T07:59:04.179 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-11T07:59:04.179 INFO:teuthology.orchestra.run.smithi122.stdout:Install 4 Packages 2024-09-11T07:59:04.179 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T07:59:04.180 INFO:teuthology.orchestra.run.smithi122.stdout:Total download size: 48 M 2024-09-11T07:59:04.180 INFO:teuthology.orchestra.run.smithi122.stdout:Installed size: 193 M 2024-09-11T07:59:04.180 INFO:teuthology.orchestra.run.smithi122.stdout:Downloading Packages: 2024-09-11T07:59:04.526 INFO:teuthology.orchestra.run.smithi122.stdout:(1/4): libcephsqlite-17.2.7-1623.gd0e6828a.el9. 723 kB/s | 163 kB 00:00 2024-09-11T07:59:04.728 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 26/26 2024-09-11T07:59:04.728 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-base-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/26 2024-09-11T07:59:04.728 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-common-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/26 2024-09-11T07:59:04.728 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-radosgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 3/26 2024-09-11T07:59:04.728 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-selinux-2:17.2.7-1623.gd0e6828a.el9.x86_64 4/26 2024-09-11T07:59:04.729 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libcephfs2-2:17.2.7-1623.gd0e6828a.el9.x86_64 5/26 2024-09-11T07:59:04.729 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libradosstriper1-2:17.2.7-1623.gd0e6828a.el9.x86_6 6/26 2024-09-11T07:59:04.729 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librgw2-2:17.2.7-1623.gd0e6828a.el9.x86_64 7/26 2024-09-11T07:59:04.729 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-ceph-argparse-2:17.2.7-1623.gd0e6828a.el9. 8/26 2024-09-11T07:59:04.729 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-ceph-common-2:17.2.7-1623.gd0e6828a.el9.x8 9/26 2024-09-11T07:59:04.729 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-cephfs-2:17.2.7-1623.gd0e6828a.el9.x86_64 10/26 2024-09-11T07:59:04.729 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-rados-2:17.2.7-1623.gd0e6828a.el9.x86_64 11/26 2024-09-11T07:59:04.729 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-rbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 12/26 2024-09-11T07:59:04.729 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-rgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 13/26 2024-09-11T07:59:04.729 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 14/26 2024-09-11T07:59:04.729 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 15/26 2024-09-11T07:59:04.729 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/26 2024-09-11T07:59:04.730 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/26 2024-09-11T07:59:04.730 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/26 2024-09-11T07:59:04.730 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 19/26 2024-09-11T07:59:04.730 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 20/26 2024-09-11T07:59:04.730 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 21/26 2024-09-11T07:59:04.730 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 22/26 2024-09-11T07:59:04.730 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librados2-2:17.2.7-1623.gd0e6828a.el9.x86_64 23/26 2024-09-11T07:59:04.730 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 24/26 2024-09-11T07:59:04.730 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librbd1-2:17.2.7-1623.gd0e6828a.el9.x86_64 25/26 2024-09-11T07:59:05.143 INFO:teuthology.orchestra.run.smithi122.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 104 kB/s | 64 kB 00:00 2024-09-11T07:59:05.218 INFO:teuthology.orchestra.run.smithi122.stdout:(3/4): socat-1.7.4.1-6.el9.x86_64.rpm 331 kB/s | 304 kB 00:00 2024-09-11T07:59:05.456 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 26/26 2024-09-11T07:59:05.456 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-11T07:59:05.456 INFO:teuthology.orchestra.run.smithi135.stdout:Upgraded: 2024-09-11T07:59:05.456 INFO:teuthology.orchestra.run.smithi135.stdout: librados2-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:59:05.456 INFO:teuthology.orchestra.run.smithi135.stdout: librbd1-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:59:05.457 INFO:teuthology.orchestra.run.smithi135.stdout:Installed: 2024-09-11T07:59:05.457 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-base-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:59:05.457 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-common-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:59:05.457 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-radosgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:59:05.457 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-selinux-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:59:05.457 INFO:teuthology.orchestra.run.smithi135.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-11T07:59:05.457 INFO:teuthology.orchestra.run.smithi135.stdout: libcephfs2-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:59:05.457 INFO:teuthology.orchestra.run.smithi135.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-11T07:59:05.457 INFO:teuthology.orchestra.run.smithi135.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-11T07:59:05.457 INFO:teuthology.orchestra.run.smithi135.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-11T07:59:05.457 INFO:teuthology.orchestra.run.smithi135.stdout: libradosstriper1-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:59:05.457 INFO:teuthology.orchestra.run.smithi135.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-11T07:59:05.457 INFO:teuthology.orchestra.run.smithi135.stdout: librgw2-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:59:05.458 INFO:teuthology.orchestra.run.smithi135.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-11T07:59:05.458 INFO:teuthology.orchestra.run.smithi135.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-11T07:59:05.458 INFO:teuthology.orchestra.run.smithi135.stdout: python3-ceph-argparse-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:59:05.458 INFO:teuthology.orchestra.run.smithi135.stdout: python3-ceph-common-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:59:05.458 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cephfs-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:59:05.458 INFO:teuthology.orchestra.run.smithi135.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-11T07:59:05.458 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rados-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:59:05.458 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:59:05.458 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:59:05.458 INFO:teuthology.orchestra.run.smithi135.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-11T07:59:05.458 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-11T07:59:05.458 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2024-09-11T07:59:05.801 INFO:teuthology.orchestra.run.smithi122.stdout:(4/4): ceph-test-17.2.7-1623.gd0e6828a.el9.x86_ 32 MB/s | 48 MB 00:01 2024-09-11T07:59:05.803 INFO:teuthology.orchestra.run.smithi122.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:59:05.804 INFO:teuthology.orchestra.run.smithi122.stdout:Total 30 MB/s | 48 MB 00:01 2024-09-11T07:59:05.831 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction check 2024-09-11T07:59:05.849 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install ceph-test 2024-09-11T07:59:05.850 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction check succeeded. 2024-09-11T07:59:05.850 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction test 2024-09-11T07:59:06.216 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction test succeeded. 2024-09-11T07:59:06.216 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction 2024-09-11T07:59:06.428 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:00:28 ago on Wed 11 Sep 2024 07:58:38 AM UTC. 2024-09-11T07:59:06.560 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2024-09-11T07:59:06.561 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-09-11T07:59:06.561 INFO:teuthology.orchestra.run.smithi135.stdout: Package Arch Version Repository Size 2024-09-11T07:59:06.561 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-09-11T07:59:06.561 INFO:teuthology.orchestra.run.smithi135.stdout:Installing: 2024-09-11T07:59:06.561 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-test x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 48 M 2024-09-11T07:59:06.561 INFO:teuthology.orchestra.run.smithi135.stdout:Installing dependencies: 2024-09-11T07:59:06.561 INFO:teuthology.orchestra.run.smithi135.stdout: libcephsqlite x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 163 k 2024-09-11T07:59:06.561 INFO:teuthology.orchestra.run.smithi135.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-11T07:59:06.561 INFO:teuthology.orchestra.run.smithi135.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-11T07:59:06.561 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-11T07:59:06.561 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction Summary 2024-09-11T07:59:06.562 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-09-11T07:59:06.562 INFO:teuthology.orchestra.run.smithi135.stdout:Install 4 Packages 2024-09-11T07:59:06.562 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-11T07:59:06.562 INFO:teuthology.orchestra.run.smithi135.stdout:Total download size: 48 M 2024-09-11T07:59:06.562 INFO:teuthology.orchestra.run.smithi135.stdout:Installed size: 193 M 2024-09-11T07:59:06.562 INFO:teuthology.orchestra.run.smithi135.stdout:Downloading Packages: 2024-09-11T07:59:06.731 INFO:teuthology.orchestra.run.smithi122.stdout: Preparing : 1/1 2024-09-11T07:59:06.813 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-11T07:59:06.859 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-11T07:59:06.887 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libcephsqlite-2:17.2.7-1623.gd0e6828a.el9.x86_64 3/4 2024-09-11T07:59:06.888 INFO:teuthology.orchestra.run.smithi135.stdout:(1/4): libcephsqlite-17.2.7-1623.gd0e6828a.el9. 722 kB/s | 163 kB 00:00 2024-09-11T07:59:07.272 INFO:teuthology.orchestra.run.smithi135.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 166 kB/s | 64 kB 00:00 2024-09-11T07:59:07.322 INFO:teuthology.orchestra.run.smithi135.stdout:(3/4): socat-1.7.4.1-6.el9.x86_64.rpm 461 kB/s | 304 kB 00:00 2024-09-11T07:59:08.199 INFO:teuthology.orchestra.run.smithi135.stdout:(4/4): ceph-test-17.2.7-1623.gd0e6828a.el9.x86_ 31 MB/s | 48 MB 00:01 2024-09-11T07:59:08.200 INFO:teuthology.orchestra.run.smithi135.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:59:08.200 INFO:teuthology.orchestra.run.smithi135.stdout:Total 30 MB/s | 48 MB 00:01 2024-09-11T07:59:08.228 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction check 2024-09-11T07:59:08.247 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction check succeeded. 2024-09-11T07:59:08.247 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction test 2024-09-11T07:59:08.614 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction test succeeded. 2024-09-11T07:59:08.615 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction 2024-09-11T07:59:09.152 INFO:teuthology.orchestra.run.smithi135.stdout: Preparing : 1/1 2024-09-11T07:59:09.233 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-11T07:59:09.272 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-11T07:59:09.300 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libcephsqlite-2:17.2.7-1623.gd0e6828a.el9.x86_64 3/4 2024-09-11T07:59:11.312 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: libcephsqlite-2:17.2.7-1623.gd0e6828a.el9.x86_64 3/4 2024-09-11T07:59:11.388 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-test-2:17.2.7-1623.gd0e6828a.el9.x86_64 4/4 2024-09-11T07:59:13.719 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: libcephsqlite-2:17.2.7-1623.gd0e6828a.el9.x86_64 3/4 2024-09-11T07:59:13.728 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-test-2:17.2.7-1623.gd0e6828a.el9.x86_64 4/4 2024-09-11T07:59:13.728 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-test-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/4 2024-09-11T07:59:13.729 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libcephsqlite-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/4 2024-09-11T07:59:13.729 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-11T07:59:13.798 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-test-2:17.2.7-1623.gd0e6828a.el9.x86_64 4/4 2024-09-11T07:59:14.023 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-11T07:59:14.023 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T07:59:14.023 INFO:teuthology.orchestra.run.smithi122.stdout:Installed: 2024-09-11T07:59:14.023 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-test-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:59:14.023 INFO:teuthology.orchestra.run.smithi122.stdout: libcephsqlite-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:59:14.023 INFO:teuthology.orchestra.run.smithi122.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-11T07:59:14.024 INFO:teuthology.orchestra.run.smithi122.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-11T07:59:14.024 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T07:59:14.024 INFO:teuthology.orchestra.run.smithi122.stdout:Complete! 2024-09-11T07:59:14.305 DEBUG:teuthology.orchestra.run.smithi122:> sudo yum -y install ceph 2024-09-11T07:59:14.868 INFO:teuthology.orchestra.run.smithi122.stdout:Last metadata expiration check: 0:00:37 ago on Wed 11 Sep 2024 07:58:37 AM UTC. 2024-09-11T07:59:15.004 INFO:teuthology.orchestra.run.smithi122.stdout:Dependencies resolved. 2024-09-11T07:59:15.006 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-11T07:59:15.006 INFO:teuthology.orchestra.run.smithi122.stdout: Package Arch Version Repository Size 2024-09-11T07:59:15.006 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-11T07:59:15.006 INFO:teuthology.orchestra.run.smithi122.stdout:Installing: 2024-09-11T07:59:15.007 INFO:teuthology.orchestra.run.smithi122.stdout: ceph x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 6.5 k 2024-09-11T07:59:15.007 INFO:teuthology.orchestra.run.smithi122.stdout:Installing dependencies: 2024-09-11T07:59:15.007 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mds x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 2.1 M 2024-09-11T07:59:15.007 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 1.4 M 2024-09-11T07:59:15.007 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-modules-core noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 240 k 2024-09-11T07:59:15.007 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mon x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 3.9 M 2024-09-11T07:59:15.007 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-osd x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 15 M 2024-09-11T07:59:15.007 INFO:teuthology.orchestra.run.smithi122.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-11T07:59:15.007 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-11T07:59:15.007 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-11T07:59:15.007 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-11T07:59:15.007 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-11T07:59:15.007 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-11T07:59:15.008 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-11T07:59:15.008 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-collections 2024-09-11T07:59:15.008 INFO:teuthology.orchestra.run.smithi122.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-09-11T07:59:15.008 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-11T07:59:15.008 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-11T07:59:15.008 INFO:teuthology.orchestra.run.smithi122.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-11T07:59:15.008 INFO:teuthology.orchestra.run.smithi122.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-11T07:59:15.008 INFO:teuthology.orchestra.run.smithi122.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-11T07:59:15.008 INFO:teuthology.orchestra.run.smithi122.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-11T07:59:15.008 INFO:teuthology.orchestra.run.smithi122.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-11T07:59:15.008 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-11T07:59:15.008 INFO:teuthology.orchestra.run.smithi122.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-11T07:59:15.008 INFO:teuthology.orchestra.run.smithi122.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-11T07:59:15.009 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-11T07:59:15.009 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-11T07:59:15.009 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-11T07:59:15.009 INFO:teuthology.orchestra.run.smithi122.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-11T07:59:15.009 INFO:teuthology.orchestra.run.smithi122.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-11T07:59:15.009 INFO:teuthology.orchestra.run.smithi122.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-11T07:59:15.009 INFO:teuthology.orchestra.run.smithi122.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-11T07:59:15.009 INFO:teuthology.orchestra.run.smithi122.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-11T07:59:15.009 INFO:teuthology.orchestra.run.smithi122.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-11T07:59:15.009 INFO:teuthology.orchestra.run.smithi122.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-11T07:59:15.009 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T07:59:15.009 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction Summary 2024-09-11T07:59:15.009 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-11T07:59:15.010 INFO:teuthology.orchestra.run.smithi122.stdout:Install 34 Packages 2024-09-11T07:59:15.010 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T07:59:15.010 INFO:teuthology.orchestra.run.smithi122.stdout:Total download size: 27 M 2024-09-11T07:59:15.010 INFO:teuthology.orchestra.run.smithi122.stdout:Installed size: 96 M 2024-09-11T07:59:15.010 INFO:teuthology.orchestra.run.smithi122.stdout:Downloading Packages: 2024-09-11T07:59:15.464 INFO:teuthology.orchestra.run.smithi122.stdout:(1/34): ceph-17.2.7-1623.gd0e6828a.el9.x86_64.r 55 kB/s | 6.5 kB 00:00 2024-09-11T07:59:15.664 INFO:teuthology.orchestra.run.smithi122.stdout:(2/34): ceph-mgr-17.2.7-1623.gd0e6828a.el9.x86_ 4.4 MB/s | 1.4 MB 00:00 2024-09-11T07:59:15.715 INFO:teuthology.orchestra.run.smithi122.stdout:(3/34): ceph-mds-17.2.7-1623.gd0e6828a.el9.x86_ 5.6 MB/s | 2.1 MB 00:00 2024-09-11T07:59:15.790 INFO:teuthology.orchestra.run.smithi122.stdout:(4/34): ceph-mon-17.2.7-1623.gd0e6828a.el9.x86_ 12 MB/s | 3.9 MB 00:00 2024-09-11T07:59:15.824 INFO:teuthology.orchestra.run.smithi122.stdout:(5/34): ceph-mgr-modules-core-17.2.7-1623.gd0e6 2.2 MB/s | 240 kB 00:00 2024-09-11T07:59:15.941 INFO:teuthology.orchestra.run.smithi122.stdout:(6/34): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.6 MB/s | 253 kB 00:00 2024-09-11T07:59:15.983 INFO:teuthology.orchestra.run.smithi122.stdout:(7/34): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-09-11T07:59:16.043 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-test-2:17.2.7-1623.gd0e6828a.el9.x86_64 4/4 2024-09-11T07:59:16.044 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-test-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/4 2024-09-11T07:59:16.044 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libcephsqlite-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/4 2024-09-11T07:59:16.044 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-11T07:59:16.058 INFO:teuthology.orchestra.run.smithi122.stdout:(8/34): python3-cryptography-36.0.1-4.el9.x86_6 5.2 MB/s | 1.2 MB 00:00 2024-09-11T07:59:16.083 INFO:teuthology.orchestra.run.smithi122.stdout:(9/34): python3-pycparser-2.20-6.el9.noarch.rpm 1.3 MB/s | 135 kB 00:00 2024-09-11T07:59:16.109 INFO:teuthology.orchestra.run.smithi122.stdout:(10/34): python3-requests-2.25.1-8.el9.noarch.r 2.4 MB/s | 125 kB 00:00 2024-09-11T07:59:16.133 INFO:teuthology.orchestra.run.smithi122.stdout:(11/34): python3-urllib3-1.26.5-6.el9.noarch.rp 4.2 MB/s | 215 kB 00:00 2024-09-11T07:59:16.159 INFO:teuthology.orchestra.run.smithi122.stdout:(12/34): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-09-11T07:59:16.184 INFO:teuthology.orchestra.run.smithi122.stdout:(13/34): python3-markupsafe-1.1.1-12.el9.x86_64 690 kB/s | 35 kB 00:00 2024-09-11T07:59:16.210 INFO:teuthology.orchestra.run.smithi122.stdout:(14/34): python3-packaging-20.9-5.el9.noarch.rp 1.5 MB/s | 77 kB 00:00 2024-09-11T07:59:16.243 INFO:teuthology.orchestra.run.smithi122.stdout:(15/34): python3-pytz-2021.1-5.el9.noarch.rpm 865 kB/s | 51 kB 00:00 2024-09-11T07:59:16.269 INFO:teuthology.orchestra.run.smithi122.stdout:(16/34): python3-toml-0.10.2-6.el9.noarch.rpm 709 kB/s | 42 kB 00:00 2024-09-11T07:59:16.360 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-11T07:59:16.360 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-11T07:59:16.360 INFO:teuthology.orchestra.run.smithi135.stdout:Installed: 2024-09-11T07:59:16.360 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-test-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:59:16.360 INFO:teuthology.orchestra.run.smithi135.stdout: libcephsqlite-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:59:16.360 INFO:teuthology.orchestra.run.smithi135.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-11T07:59:16.360 INFO:teuthology.orchestra.run.smithi135.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-11T07:59:16.360 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-11T07:59:16.360 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2024-09-11T07:59:16.446 INFO:teuthology.orchestra.run.smithi122.stdout:(17/34): ceph-osd-17.2.7-1623.gd0e6828a.el9.x86 20 MB/s | 15 MB 00:00 2024-09-11T07:59:16.462 INFO:teuthology.orchestra.run.smithi122.stdout:(18/34): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 199 kB/s | 43 kB 00:00 2024-09-11T07:59:16.505 INFO:teuthology.orchestra.run.smithi122.stdout:(19/34): python3-cheroot-8.6.0-4.el9.noarch.rpm 730 kB/s | 172 kB 00:00 2024-09-11T07:59:16.530 INFO:teuthology.orchestra.run.smithi122.stdout:(20/34): python3-cherrypy-18.6.1-2.el9.noarch.r 5.4 MB/s | 358 kB 00:00 2024-09-11T07:59:16.547 INFO:teuthology.orchestra.run.smithi122.stdout:(21/34): python3-jaraco-8.2.1-3.el9.noarch.rpm 130 kB/s | 11 kB 00:00 2024-09-11T07:59:16.563 INFO:teuthology.orchestra.run.smithi122.stdout:(22/34): python3-jaraco-classes-3.2.1-5.el9.noa 311 kB/s | 18 kB 00:00 2024-09-11T07:59:16.580 INFO:teuthology.orchestra.run.smithi122.stdout:(23/34): python3-jaraco-collections-3.0.0-8.el9 459 kB/s | 23 kB 00:00 2024-09-11T07:59:16.597 INFO:teuthology.orchestra.run.smithi122.stdout:(24/34): python3-jaraco-functools-3.5.0-2.el9.n 579 kB/s | 19 kB 00:00 2024-09-11T07:59:16.614 INFO:teuthology.orchestra.run.smithi122.stdout:(25/34): python3-jaraco-text-3.2.0-6.el9.noarch 390 kB/s | 20 kB 00:00 2024-09-11T07:59:16.631 INFO:teuthology.orchestra.run.smithi122.stdout:(26/34): python3-logutils-0.3.5-21.el9.noarch.r 918 kB/s | 46 kB 00:00 2024-09-11T07:59:16.649 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install ceph 2024-09-11T07:59:16.656 INFO:teuthology.orchestra.run.smithi122.stdout:(27/34): python3-more-itertools-8.12.0-2.el9.no 1.8 MB/s | 79 kB 00:00 2024-09-11T07:59:16.673 INFO:teuthology.orchestra.run.smithi122.stdout:(28/34): python3-portend-3.1.0-2.el9.noarch.rpm 393 kB/s | 16 kB 00:00 2024-09-11T07:59:16.698 INFO:teuthology.orchestra.run.smithi122.stdout:(29/34): python3-pyOpenSSL-21.0.0-1.el9.noarch. 2.1 MB/s | 90 kB 00:00 2024-09-11T07:59:16.716 INFO:teuthology.orchestra.run.smithi122.stdout:(30/34): python3-tempora-5.0.0-2.el9.noarch.rpm 851 kB/s | 36 kB 00:00 2024-09-11T07:59:16.750 INFO:teuthology.orchestra.run.smithi122.stdout:(31/34): python3-pecan-1.4.2-3.el9.noarch.rpm 2.0 MB/s | 272 kB 00:00 2024-09-11T07:59:16.766 INFO:teuthology.orchestra.run.smithi122.stdout:(32/34): python3-zc-lockfile-2.0-10.el9.noarch. 1.2 MB/s | 20 kB 00:00 2024-09-11T07:59:16.792 INFO:teuthology.orchestra.run.smithi122.stdout:(33/34): python3-webob-1.8.8-2.el9.noarch.rpm 2.4 MB/s | 230 kB 00:00 2024-09-11T07:59:16.816 INFO:teuthology.orchestra.run.smithi122.stdout:(34/34): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.2 MB/s | 427 kB 00:00 2024-09-11T07:59:16.821 INFO:teuthology.orchestra.run.smithi122.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:59:16.821 INFO:teuthology.orchestra.run.smithi122.stdout:Total 15 MB/s | 27 MB 00:01 2024-09-11T07:59:16.947 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction check 2024-09-11T07:59:16.985 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction check succeeded. 2024-09-11T07:59:16.985 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction test 2024-09-11T07:59:17.226 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:00:39 ago on Wed 11 Sep 2024 07:58:38 AM UTC. 2024-09-11T07:59:17.362 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction test succeeded. 2024-09-11T07:59:17.363 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction 2024-09-11T07:59:17.365 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2024-09-11T07:59:17.367 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-09-11T07:59:17.367 INFO:teuthology.orchestra.run.smithi135.stdout: Package Arch Version Repository Size 2024-09-11T07:59:17.367 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-09-11T07:59:17.367 INFO:teuthology.orchestra.run.smithi135.stdout:Installing: 2024-09-11T07:59:17.367 INFO:teuthology.orchestra.run.smithi135.stdout: ceph x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 6.5 k 2024-09-11T07:59:17.368 INFO:teuthology.orchestra.run.smithi135.stdout:Installing dependencies: 2024-09-11T07:59:17.368 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mds x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 2.1 M 2024-09-11T07:59:17.368 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 1.4 M 2024-09-11T07:59:17.368 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-modules-core noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 240 k 2024-09-11T07:59:17.368 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mon x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 3.9 M 2024-09-11T07:59:17.368 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-osd x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 15 M 2024-09-11T07:59:17.368 INFO:teuthology.orchestra.run.smithi135.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-11T07:59:17.368 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-11T07:59:17.368 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-11T07:59:17.368 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-11T07:59:17.368 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-11T07:59:17.368 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-11T07:59:17.368 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-11T07:59:17.369 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-collections 2024-09-11T07:59:17.369 INFO:teuthology.orchestra.run.smithi135.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-09-11T07:59:17.369 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-11T07:59:17.369 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-11T07:59:17.369 INFO:teuthology.orchestra.run.smithi135.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-11T07:59:17.369 INFO:teuthology.orchestra.run.smithi135.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-11T07:59:17.369 INFO:teuthology.orchestra.run.smithi135.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-11T07:59:17.369 INFO:teuthology.orchestra.run.smithi135.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-11T07:59:17.369 INFO:teuthology.orchestra.run.smithi135.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-11T07:59:17.369 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-11T07:59:17.369 INFO:teuthology.orchestra.run.smithi135.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-11T07:59:17.369 INFO:teuthology.orchestra.run.smithi135.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-11T07:59:17.369 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-11T07:59:17.369 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-11T07:59:17.369 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-11T07:59:17.370 INFO:teuthology.orchestra.run.smithi135.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-11T07:59:17.370 INFO:teuthology.orchestra.run.smithi135.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-11T07:59:17.370 INFO:teuthology.orchestra.run.smithi135.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-11T07:59:17.370 INFO:teuthology.orchestra.run.smithi135.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-11T07:59:17.370 INFO:teuthology.orchestra.run.smithi135.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-11T07:59:17.370 INFO:teuthology.orchestra.run.smithi135.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-11T07:59:17.370 INFO:teuthology.orchestra.run.smithi135.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-11T07:59:17.370 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-11T07:59:17.370 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction Summary 2024-09-11T07:59:17.370 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-09-11T07:59:17.370 INFO:teuthology.orchestra.run.smithi135.stdout:Install 34 Packages 2024-09-11T07:59:17.370 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-11T07:59:17.371 INFO:teuthology.orchestra.run.smithi135.stdout:Total download size: 27 M 2024-09-11T07:59:17.371 INFO:teuthology.orchestra.run.smithi135.stdout:Installed size: 96 M 2024-09-11T07:59:17.371 INFO:teuthology.orchestra.run.smithi135.stdout:Downloading Packages: 2024-09-11T07:59:17.977 INFO:teuthology.orchestra.run.smithi135.stdout:(1/34): ceph-17.2.7-1623.gd0e6828a.el9.x86_64.r 52 kB/s | 6.5 kB 00:00 2024-09-11T07:59:18.160 INFO:teuthology.orchestra.run.smithi135.stdout:(2/34): ceph-mgr-17.2.7-1623.gd0e6828a.el9.x86_ 4.6 MB/s | 1.4 MB 00:00 2024-09-11T07:59:18.219 INFO:teuthology.orchestra.run.smithi135.stdout:(3/34): ceph-mds-17.2.7-1623.gd0e6828a.el9.x86_ 5.6 MB/s | 2.1 MB 00:00 2024-09-11T07:59:18.312 INFO:teuthology.orchestra.run.smithi135.stdout:(4/34): ceph-mgr-modules-core-17.2.7-1623.gd0e6 2.6 MB/s | 240 kB 00:00 2024-09-11T07:59:18.378 INFO:teuthology.orchestra.run.smithi135.stdout:(5/34): ceph-mon-17.2.7-1623.gd0e6828a.el9.x86_ 9.8 MB/s | 3.9 MB 00:00 2024-09-11T07:59:18.495 INFO:teuthology.orchestra.run.smithi135.stdout:(6/34): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.3 MB/s | 253 kB 00:00 2024-09-11T07:59:18.537 INFO:teuthology.orchestra.run.smithi135.stdout:(7/34): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-09-11T07:59:18.579 INFO:teuthology.orchestra.run.smithi135.stdout:(8/34): python3-cryptography-36.0.1-4.el9.x86_6 6.1 MB/s | 1.2 MB 00:00 2024-09-11T07:59:18.604 INFO:teuthology.orchestra.run.smithi135.stdout:(9/34): python3-pycparser-2.20-6.el9.noarch.rpm 2.0 MB/s | 135 kB 00:00 2024-09-11T07:59:18.619 INFO:teuthology.orchestra.run.smithi122.stdout: Preparing : 1/1 2024-09-11T07:59:18.629 INFO:teuthology.orchestra.run.smithi135.stdout:(10/34): python3-requests-2.25.1-8.el9.noarch.r 2.4 MB/s | 125 kB 00:00 2024-09-11T07:59:18.663 INFO:teuthology.orchestra.run.smithi135.stdout:(11/34): python3-urllib3-1.26.5-6.el9.noarch.rp 3.6 MB/s | 215 kB 00:00 2024-09-11T07:59:18.686 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/34 2024-09-11T07:59:18.718 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/34 2024-09-11T07:59:18.751 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/34 2024-09-11T07:59:18.830 INFO:teuthology.orchestra.run.smithi135.stdout:(12/34): ceph-osd-17.2.7-1623.gd0e6828a.el9.x86 23 MB/s | 15 MB 00:00 2024-09-11T07:59:18.843 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/34 2024-09-11T07:59:18.905 INFO:teuthology.orchestra.run.smithi135.stdout:(13/34): python3-markupsafe-1.1.1-12.el9.x86_64 144 kB/s | 35 kB 00:00 2024-09-11T07:59:18.922 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/34 2024-09-11T07:59:18.939 INFO:teuthology.orchestra.run.smithi135.stdout:(14/34): python3-mako-1.1.4-6.el9.noarch.rpm 557 kB/s | 172 kB 00:00 2024-09-11T07:59:18.964 INFO:teuthology.orchestra.run.smithi135.stdout:(15/34): python3-packaging-20.9-5.el9.noarch.rp 577 kB/s | 77 kB 00:00 2024-09-11T07:59:18.971 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 6/34 2024-09-11T07:59:18.981 INFO:teuthology.orchestra.run.smithi135.stdout:(16/34): python3-pytz-2021.1-5.el9.noarch.rpm 675 kB/s | 51 kB 00:00 2024-09-11T07:59:18.998 INFO:teuthology.orchestra.run.smithi135.stdout:(17/34): python3-toml-0.10.2-6.el9.noarch.rpm 707 kB/s | 42 kB 00:00 2024-09-11T07:59:19.014 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/34 2024-09-11T07:59:19.040 INFO:teuthology.orchestra.run.smithi135.stdout:(18/34): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 577 kB/s | 43 kB 00:00 2024-09-11T07:59:19.044 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/34 2024-09-11T07:59:19.057 INFO:teuthology.orchestra.run.smithi135.stdout:(19/34): python3-jaraco-8.2.1-3.el9.noarch.rpm 718 kB/s | 11 kB 00:00 2024-09-11T07:59:19.078 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/34 2024-09-11T07:59:19.082 INFO:teuthology.orchestra.run.smithi135.stdout:(20/34): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.7 MB/s | 172 kB 00:00 2024-09-11T07:59:19.099 INFO:teuthology.orchestra.run.smithi135.stdout:(21/34): python3-jaraco-classes-3.2.1-5.el9.noa 422 kB/s | 18 kB 00:00 2024-09-11T07:59:19.112 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/34 2024-09-11T07:59:19.116 INFO:teuthology.orchestra.run.smithi135.stdout:(22/34): python3-jaraco-collections-3.0.0-8.el9 692 kB/s | 23 kB 00:00 2024-09-11T07:59:19.133 INFO:teuthology.orchestra.run.smithi135.stdout:(23/34): python3-jaraco-functools-3.5.0-2.el9.n 579 kB/s | 19 kB 00:00 2024-09-11T07:59:19.150 INFO:teuthology.orchestra.run.smithi135.stdout:(24/34): python3-jaraco-text-3.2.0-6.el9.noarch 585 kB/s | 20 kB 00:00 2024-09-11T07:59:19.152 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/34 2024-09-11T07:59:19.167 INFO:teuthology.orchestra.run.smithi135.stdout:(25/34): python3-logutils-0.3.5-21.el9.noarch.r 1.4 MB/s | 46 kB 00:00 2024-09-11T07:59:19.187 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/34 2024-09-11T07:59:19.192 INFO:teuthology.orchestra.run.smithi135.stdout:(26/34): python3-more-itertools-8.12.0-2.el9.no 1.8 MB/s | 79 kB 00:00 2024-09-11T07:59:19.221 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/34 2024-09-11T07:59:19.226 INFO:teuthology.orchestra.run.smithi135.stdout:(27/34): python3-cherrypy-18.6.1-2.el9.noarch.r 1.6 MB/s | 358 kB 00:00 2024-09-11T07:59:19.243 INFO:teuthology.orchestra.run.smithi135.stdout:(28/34): python3-portend-3.1.0-2.el9.noarch.rpm 326 kB/s | 16 kB 00:00 2024-09-11T07:59:19.276 INFO:teuthology.orchestra.run.smithi135.stdout:(29/34): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.8 MB/s | 90 kB 00:00 2024-09-11T07:59:19.278 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/34 2024-09-11T07:59:19.293 INFO:teuthology.orchestra.run.smithi135.stdout:(30/34): python3-tempora-5.0.0-2.el9.noarch.rpm 715 kB/s | 36 kB 00:00 2024-09-11T07:59:19.319 INFO:teuthology.orchestra.run.smithi135.stdout:(31/34): python3-pecan-1.4.2-3.el9.noarch.rpm 1.8 MB/s | 272 kB 00:00 2024-09-11T07:59:19.319 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-packaging-20.9-5.el9.noarch 15/34 2024-09-11T07:59:19.352 INFO:teuthology.orchestra.run.smithi135.stdout:(32/34): python3-zc-lockfile-2.0-10.el9.noarch. 601 kB/s | 20 kB 00:00 2024-09-11T07:59:19.377 INFO:teuthology.orchestra.run.smithi135.stdout:(33/34): python3-webob-1.8.8-2.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-09-11T07:59:19.391 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 16/34 2024-09-11T07:59:19.411 INFO:teuthology.orchestra.run.smithi135.stdout:(34/34): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.6 MB/s | 427 kB 00:00 2024-09-11T07:59:19.416 INFO:teuthology.orchestra.run.smithi135.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:59:19.416 INFO:teuthology.orchestra.run.smithi135.stdout:Total 13 MB/s | 27 MB 00:02 2024-09-11T07:59:19.530 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 17/34 2024-09-11T07:59:19.547 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction check 2024-09-11T07:59:19.584 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction check succeeded. 2024-09-11T07:59:19.584 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction test 2024-09-11T07:59:19.611 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 18/34 2024-09-11T07:59:19.664 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 19/34 2024-09-11T07:59:19.702 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 20/34 2024-09-11T07:59:19.768 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-ply-3.11-14.el9.noarch 21/34 2024-09-11T07:59:19.823 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 22/34 2024-09-11T07:59:19.987 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction test succeeded. 2024-09-11T07:59:19.987 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction 2024-09-11T07:59:20.010 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 23/34 2024-09-11T07:59:20.065 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 24/34 2024-09-11T07:59:20.136 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 25/34 2024-09-11T07:59:20.224 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 26/34 2024-09-11T07:59:20.265 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 27/34 2024-09-11T07:59:20.381 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 28/34 2024-09-11T07:59:20.537 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-1623.gd0e6828a.el9. 29/34 2024-09-11T07:59:20.568 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mgr-2:17.2.7-1623.gd0e6828a.el9.x86_64 30/34 2024-09-11T07:59:20.594 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-mgr-2:17.2.7-1623.gd0e6828a.el9.x86_64 30/34 2024-09-11T07:59:20.595 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T07:59:20.595 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-11T07:59:20.595 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T07:59:20.595 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T07:59:20.595 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T07:59:21.257 INFO:teuthology.orchestra.run.smithi135.stdout: Preparing : 1/1 2024-09-11T07:59:21.322 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/34 2024-09-11T07:59:21.354 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/34 2024-09-11T07:59:21.396 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/34 2024-09-11T07:59:21.488 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/34 2024-09-11T07:59:21.566 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/34 2024-09-11T07:59:21.615 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 6/34 2024-09-11T07:59:21.667 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/34 2024-09-11T07:59:21.697 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/34 2024-09-11T07:59:21.732 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/34 2024-09-11T07:59:21.766 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/34 2024-09-11T07:59:21.816 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/34 2024-09-11T07:59:21.866 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/34 2024-09-11T07:59:21.900 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/34 2024-09-11T07:59:21.949 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/34 2024-09-11T07:59:21.981 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-packaging-20.9-5.el9.noarch 15/34 2024-09-11T07:59:21.986 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-osd-2:17.2.7-1623.gd0e6828a.el9.x86_64 31/34 2024-09-11T07:59:22.016 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-osd-2:17.2.7-1623.gd0e6828a.el9.x86_64 31/34 2024-09-11T07:59:22.016 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T07:59:22.016 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-11T07:59:22.016 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T07:59:22.016 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T07:59:22.016 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T07:59:22.061 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 16/34 2024-09-11T07:59:22.202 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 17/34 2024-09-11T07:59:22.281 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 18/34 2024-09-11T07:59:22.334 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 19/34 2024-09-11T07:59:22.372 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 20/34 2024-09-11T07:59:22.395 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mon-2:17.2.7-1623.gd0e6828a.el9.x86_64 32/34 2024-09-11T07:59:22.419 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-mon-2:17.2.7-1623.gd0e6828a.el9.x86_64 32/34 2024-09-11T07:59:22.419 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T07:59:22.419 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-11T07:59:22.420 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T07:59:22.420 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T07:59:22.420 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T07:59:22.422 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-ply-3.11-14.el9.noarch 21/34 2024-09-11T07:59:22.485 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 22/34 2024-09-11T07:59:22.621 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mds-2:17.2.7-1623.gd0e6828a.el9.x86_64 33/34 2024-09-11T07:59:22.645 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-mds-2:17.2.7-1623.gd0e6828a.el9.x86_64 33/34 2024-09-11T07:59:22.645 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T07:59:22.645 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-11T07:59:22.645 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T07:59:22.645 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T07:59:22.645 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T07:59:22.682 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 23/34 2024-09-11T07:59:22.710 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-2:17.2.7-1623.gd0e6828a.el9.x86_64 34/34 2024-09-11T07:59:22.734 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 24/34 2024-09-11T07:59:22.812 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 25/34 2024-09-11T07:59:22.903 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 26/34 2024-09-11T07:59:22.984 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 27/34 2024-09-11T07:59:23.119 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 28/34 2024-09-11T07:59:23.280 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-1623.gd0e6828a.el9. 29/34 2024-09-11T07:59:23.319 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mgr-2:17.2.7-1623.gd0e6828a.el9.x86_64 30/34 2024-09-11T07:59:23.345 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-mgr-2:17.2.7-1623.gd0e6828a.el9.x86_64 30/34 2024-09-11T07:59:23.345 INFO:teuthology.orchestra.run.smithi135.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T07:59:23.346 INFO:teuthology.orchestra.run.smithi135.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-11T07:59:23.346 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T07:59:23.346 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T07:59:23.346 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-11T07:59:24.522 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-2:17.2.7-1623.gd0e6828a.el9.x86_64 34/34 2024-09-11T07:59:24.522 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/34 2024-09-11T07:59:24.522 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mds-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/34 2024-09-11T07:59:24.522 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mgr-2:17.2.7-1623.gd0e6828a.el9.x86_64 3/34 2024-09-11T07:59:24.522 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mon-2:17.2.7-1623.gd0e6828a.el9.x86_64 4/34 2024-09-11T07:59:24.522 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-osd-2:17.2.7-1623.gd0e6828a.el9.x86_64 5/34 2024-09-11T07:59:24.522 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-1623.gd0e6828a.el9. 6/34 2024-09-11T07:59:24.523 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/34 2024-09-11T07:59:24.523 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/34 2024-09-11T07:59:24.523 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/34 2024-09-11T07:59:24.523 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/34 2024-09-11T07:59:24.523 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/34 2024-09-11T07:59:24.523 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/34 2024-09-11T07:59:24.523 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/34 2024-09-11T07:59:24.523 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/34 2024-09-11T07:59:24.523 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 15/34 2024-09-11T07:59:24.523 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 16/34 2024-09-11T07:59:24.523 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 17/34 2024-09-11T07:59:24.523 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 18/34 2024-09-11T07:59:24.523 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 19/34 2024-09-11T07:59:24.523 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 20/34 2024-09-11T07:59:24.524 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 21/34 2024-09-11T07:59:24.524 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 22/34 2024-09-11T07:59:24.524 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 23/34 2024-09-11T07:59:24.524 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 24/34 2024-09-11T07:59:24.524 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 25/34 2024-09-11T07:59:24.524 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 26/34 2024-09-11T07:59:24.524 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 27/34 2024-09-11T07:59:24.524 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 28/34 2024-09-11T07:59:24.525 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 29/34 2024-09-11T07:59:24.525 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 30/34 2024-09-11T07:59:24.525 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 31/34 2024-09-11T07:59:24.525 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 32/34 2024-09-11T07:59:24.525 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 33/34 2024-09-11T07:59:24.670 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-osd-2:17.2.7-1623.gd0e6828a.el9.x86_64 31/34 2024-09-11T07:59:24.698 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-osd-2:17.2.7-1623.gd0e6828a.el9.x86_64 31/34 2024-09-11T07:59:24.698 INFO:teuthology.orchestra.run.smithi135.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T07:59:24.698 INFO:teuthology.orchestra.run.smithi135.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-11T07:59:24.698 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T07:59:24.699 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T07:59:24.699 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-11T07:59:25.097 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mon-2:17.2.7-1623.gd0e6828a.el9.x86_64 32/34 2024-09-11T07:59:25.130 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-mon-2:17.2.7-1623.gd0e6828a.el9.x86_64 32/34 2024-09-11T07:59:25.130 INFO:teuthology.orchestra.run.smithi135.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T07:59:25.130 INFO:teuthology.orchestra.run.smithi135.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-11T07:59:25.130 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T07:59:25.130 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T07:59:25.130 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-11T07:59:25.339 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mds-2:17.2.7-1623.gd0e6828a.el9.x86_64 33/34 2024-09-11T07:59:25.365 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-mds-2:17.2.7-1623.gd0e6828a.el9.x86_64 33/34 2024-09-11T07:59:25.365 INFO:teuthology.orchestra.run.smithi135.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T07:59:25.365 INFO:teuthology.orchestra.run.smithi135.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-11T07:59:25.365 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T07:59:25.365 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T07:59:25.365 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-11T07:59:25.413 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 34/34 2024-09-11T07:59:25.413 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T07:59:25.413 INFO:teuthology.orchestra.run.smithi122.stdout:Installed: 2024-09-11T07:59:25.413 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:59:25.413 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mds-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:59:25.413 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:59:25.414 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-modules-core-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T07:59:25.414 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mon-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:59:25.414 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-osd-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:59:25.414 INFO:teuthology.orchestra.run.smithi122.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-11T07:59:25.414 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-11T07:59:25.414 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-11T07:59:25.414 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-11T07:59:25.414 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-11T07:59:25.414 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-11T07:59:25.414 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-11T07:59:25.414 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-11T07:59:25.414 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-11T07:59:25.415 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-11T07:59:25.415 INFO:teuthology.orchestra.run.smithi122.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-11T07:59:25.415 INFO:teuthology.orchestra.run.smithi122.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-11T07:59:25.415 INFO:teuthology.orchestra.run.smithi122.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-11T07:59:25.415 INFO:teuthology.orchestra.run.smithi122.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-11T07:59:25.415 INFO:teuthology.orchestra.run.smithi122.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-11T07:59:25.415 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-11T07:59:25.415 INFO:teuthology.orchestra.run.smithi122.stdout: python3-ply-3.11-14.el9.noarch 2024-09-11T07:59:25.415 INFO:teuthology.orchestra.run.smithi122.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-11T07:59:25.415 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-11T07:59:25.415 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-11T07:59:25.415 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-11T07:59:25.415 INFO:teuthology.orchestra.run.smithi122.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-11T07:59:25.416 INFO:teuthology.orchestra.run.smithi122.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-11T07:59:25.416 INFO:teuthology.orchestra.run.smithi122.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-11T07:59:25.416 INFO:teuthology.orchestra.run.smithi122.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-11T07:59:25.416 INFO:teuthology.orchestra.run.smithi122.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-11T07:59:25.416 INFO:teuthology.orchestra.run.smithi122.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-11T07:59:25.416 INFO:teuthology.orchestra.run.smithi122.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-11T07:59:25.416 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T07:59:25.416 INFO:teuthology.orchestra.run.smithi122.stdout:Complete! 2024-09-11T07:59:25.431 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-2:17.2.7-1623.gd0e6828a.el9.x86_64 34/34 2024-09-11T07:59:25.804 DEBUG:teuthology.orchestra.run.smithi122:> sudo yum -y install ceph-base 2024-09-11T07:59:26.367 INFO:teuthology.orchestra.run.smithi122.stdout:Last metadata expiration check: 0:00:49 ago on Wed 11 Sep 2024 07:58:37 AM UTC. 2024-09-11T07:59:26.457 INFO:teuthology.orchestra.run.smithi122.stdout:Package ceph-base-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T07:59:26.498 INFO:teuthology.orchestra.run.smithi122.stdout:Dependencies resolved. 2024-09-11T07:59:26.498 INFO:teuthology.orchestra.run.smithi122.stdout:Nothing to do. 2024-09-11T07:59:26.499 INFO:teuthology.orchestra.run.smithi122.stdout:Complete! 2024-09-11T07:59:26.574 DEBUG:teuthology.orchestra.run.smithi122:> sudo yum -y install cephadm 2024-09-11T07:59:27.129 INFO:teuthology.orchestra.run.smithi122.stdout:Last metadata expiration check: 0:00:50 ago on Wed 11 Sep 2024 07:58:37 AM UTC. 2024-09-11T07:59:27.256 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-2:17.2.7-1623.gd0e6828a.el9.x86_64 34/34 2024-09-11T07:59:27.256 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/34 2024-09-11T07:59:27.256 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mds-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/34 2024-09-11T07:59:27.256 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mgr-2:17.2.7-1623.gd0e6828a.el9.x86_64 3/34 2024-09-11T07:59:27.256 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mon-2:17.2.7-1623.gd0e6828a.el9.x86_64 4/34 2024-09-11T07:59:27.256 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-osd-2:17.2.7-1623.gd0e6828a.el9.x86_64 5/34 2024-09-11T07:59:27.257 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-1623.gd0e6828a.el9. 6/34 2024-09-11T07:59:27.257 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/34 2024-09-11T07:59:27.257 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/34 2024-09-11T07:59:27.257 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/34 2024-09-11T07:59:27.257 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/34 2024-09-11T07:59:27.257 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/34 2024-09-11T07:59:27.257 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/34 2024-09-11T07:59:27.257 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/34 2024-09-11T07:59:27.257 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/34 2024-09-11T07:59:27.257 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 15/34 2024-09-11T07:59:27.257 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 16/34 2024-09-11T07:59:27.257 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 17/34 2024-09-11T07:59:27.257 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 18/34 2024-09-11T07:59:27.258 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 19/34 2024-09-11T07:59:27.258 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 20/34 2024-09-11T07:59:27.258 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 21/34 2024-09-11T07:59:27.258 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 22/34 2024-09-11T07:59:27.258 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 23/34 2024-09-11T07:59:27.258 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 24/34 2024-09-11T07:59:27.258 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 25/34 2024-09-11T07:59:27.258 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 26/34 2024-09-11T07:59:27.258 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 27/34 2024-09-11T07:59:27.258 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 28/34 2024-09-11T07:59:27.259 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 29/34 2024-09-11T07:59:27.259 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 30/34 2024-09-11T07:59:27.259 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 31/34 2024-09-11T07:59:27.260 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 32/34 2024-09-11T07:59:27.260 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 33/34 2024-09-11T07:59:27.260 INFO:teuthology.orchestra.run.smithi122.stdout:Dependencies resolved. 2024-09-11T07:59:27.260 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-11T07:59:27.260 INFO:teuthology.orchestra.run.smithi122.stdout: Package Arch Version Repository Size 2024-09-11T07:59:27.260 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-11T07:59:27.260 INFO:teuthology.orchestra.run.smithi122.stdout:Installing: 2024-09-11T07:59:27.261 INFO:teuthology.orchestra.run.smithi122.stdout: cephadm noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 83 k 2024-09-11T07:59:27.261 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T07:59:27.261 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction Summary 2024-09-11T07:59:27.261 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-11T07:59:27.261 INFO:teuthology.orchestra.run.smithi122.stdout:Install 1 Package 2024-09-11T07:59:27.261 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T07:59:27.261 INFO:teuthology.orchestra.run.smithi122.stdout:Total download size: 83 k 2024-09-11T07:59:27.261 INFO:teuthology.orchestra.run.smithi122.stdout:Installed size: 375 k 2024-09-11T07:59:27.261 INFO:teuthology.orchestra.run.smithi122.stdout:Downloading Packages: 2024-09-11T07:59:27.454 INFO:teuthology.orchestra.run.smithi122.stdout:cephadm-17.2.7-1623.gd0e6828a.el9.noarch.rpm 428 kB/s | 83 kB 00:00 2024-09-11T07:59:27.454 INFO:teuthology.orchestra.run.smithi122.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:59:27.454 INFO:teuthology.orchestra.run.smithi122.stdout:Total 425 kB/s | 83 kB 00:00 2024-09-11T07:59:27.454 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction check 2024-09-11T07:59:27.460 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction check succeeded. 2024-09-11T07:59:27.460 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction test 2024-09-11T07:59:27.464 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction test succeeded. 2024-09-11T07:59:27.464 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction 2024-09-11T07:59:27.536 INFO:teuthology.orchestra.run.smithi122.stdout: Preparing : 1/1 2024-09-11T07:59:27.970 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 1/1 2024-09-11T07:59:28.049 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 1/1 2024-09-11T07:59:28.260 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 34/34 2024-09-11T07:59:28.261 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-11T07:59:28.261 INFO:teuthology.orchestra.run.smithi135.stdout:Installed: 2024-09-11T07:59:28.261 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:59:28.261 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mds-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:59:28.261 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:59:28.261 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-modules-core-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T07:59:28.261 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mon-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:59:28.261 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-osd-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:59:28.261 INFO:teuthology.orchestra.run.smithi135.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-11T07:59:28.261 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-11T07:59:28.261 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-11T07:59:28.261 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-11T07:59:28.262 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-11T07:59:28.262 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-11T07:59:28.262 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-11T07:59:28.262 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-11T07:59:28.262 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-11T07:59:28.262 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-11T07:59:28.262 INFO:teuthology.orchestra.run.smithi135.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-11T07:59:28.262 INFO:teuthology.orchestra.run.smithi135.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-11T07:59:28.262 INFO:teuthology.orchestra.run.smithi135.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-11T07:59:28.262 INFO:teuthology.orchestra.run.smithi135.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-11T07:59:28.262 INFO:teuthology.orchestra.run.smithi135.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-11T07:59:28.262 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-11T07:59:28.263 INFO:teuthology.orchestra.run.smithi135.stdout: python3-ply-3.11-14.el9.noarch 2024-09-11T07:59:28.263 INFO:teuthology.orchestra.run.smithi135.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-11T07:59:28.263 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-11T07:59:28.263 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-11T07:59:28.263 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-11T07:59:28.263 INFO:teuthology.orchestra.run.smithi135.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-11T07:59:28.263 INFO:teuthology.orchestra.run.smithi135.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-11T07:59:28.263 INFO:teuthology.orchestra.run.smithi135.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-11T07:59:28.263 INFO:teuthology.orchestra.run.smithi135.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-11T07:59:28.263 INFO:teuthology.orchestra.run.smithi135.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-11T07:59:28.263 INFO:teuthology.orchestra.run.smithi135.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-11T07:59:28.263 INFO:teuthology.orchestra.run.smithi135.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-11T07:59:28.263 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-11T07:59:28.264 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2024-09-11T07:59:28.396 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 1/1 2024-09-11T07:59:28.636 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 1/1 2024-09-11T07:59:28.636 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T07:59:28.636 INFO:teuthology.orchestra.run.smithi122.stdout:Installed: 2024-09-11T07:59:28.636 INFO:teuthology.orchestra.run.smithi122.stdout: cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T07:59:28.636 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T07:59:28.636 INFO:teuthology.orchestra.run.smithi122.stdout:Complete! 2024-09-11T07:59:28.656 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install ceph-base 2024-09-11T07:59:28.867 DEBUG:teuthology.orchestra.run.smithi122:> sudo yum -y install ceph-immutable-object-cache 2024-09-11T07:59:29.228 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:00:51 ago on Wed 11 Sep 2024 07:58:38 AM UTC. 2024-09-11T07:59:29.318 INFO:teuthology.orchestra.run.smithi135.stdout:Package ceph-base-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T07:59:29.360 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2024-09-11T07:59:29.361 INFO:teuthology.orchestra.run.smithi135.stdout:Nothing to do. 2024-09-11T07:59:29.361 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2024-09-11T07:59:29.423 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install cephadm 2024-09-11T07:59:29.427 INFO:teuthology.orchestra.run.smithi122.stdout:Last metadata expiration check: 0:00:52 ago on Wed 11 Sep 2024 07:58:37 AM UTC. 2024-09-11T07:59:29.563 INFO:teuthology.orchestra.run.smithi122.stdout:Dependencies resolved. 2024-09-11T07:59:29.563 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-11T07:59:29.563 INFO:teuthology.orchestra.run.smithi122.stdout: Package Arch Version Repo Size 2024-09-11T07:59:29.564 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-11T07:59:29.564 INFO:teuthology.orchestra.run.smithi122.stdout:Installing: 2024-09-11T07:59:29.564 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 137 k 2024-09-11T07:59:29.564 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T07:59:29.564 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction Summary 2024-09-11T07:59:29.564 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-11T07:59:29.564 INFO:teuthology.orchestra.run.smithi122.stdout:Install 1 Package 2024-09-11T07:59:29.564 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T07:59:29.564 INFO:teuthology.orchestra.run.smithi122.stdout:Total download size: 137 k 2024-09-11T07:59:29.564 INFO:teuthology.orchestra.run.smithi122.stdout:Installed size: 379 k 2024-09-11T07:59:29.564 INFO:teuthology.orchestra.run.smithi122.stdout:Downloading Packages: 2024-09-11T07:59:29.769 INFO:teuthology.orchestra.run.smithi122.stdout:ceph-immutable-object-cache-17.2.7-1623.gd0e682 673 kB/s | 137 kB 00:00 2024-09-11T07:59:29.769 INFO:teuthology.orchestra.run.smithi122.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:59:29.770 INFO:teuthology.orchestra.run.smithi122.stdout:Total 667 kB/s | 137 kB 00:00 2024-09-11T07:59:29.770 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction check 2024-09-11T07:59:29.777 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction check succeeded. 2024-09-11T07:59:29.777 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction test 2024-09-11T07:59:29.833 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction test succeeded. 2024-09-11T07:59:29.834 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction 2024-09-11T07:59:29.986 INFO:teuthology.orchestra.run.smithi122.stdout: Preparing : 1/1 2024-09-11T07:59:29.990 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:00:51 ago on Wed 11 Sep 2024 07:58:38 AM UTC. 2024-09-11T07:59:30.053 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-1623.gd0e6828a. 1/1 2024-09-11T07:59:30.078 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-1623.gd0e6828a. 1/1 2024-09-11T07:59:30.078 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T07:59:30.079 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-11T07:59:30.079 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T07:59:30.124 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2024-09-11T07:59:30.124 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-09-11T07:59:30.125 INFO:teuthology.orchestra.run.smithi135.stdout: Package Arch Version Repository Size 2024-09-11T07:59:30.125 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-09-11T07:59:30.125 INFO:teuthology.orchestra.run.smithi135.stdout:Installing: 2024-09-11T07:59:30.125 INFO:teuthology.orchestra.run.smithi135.stdout: cephadm noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 83 k 2024-09-11T07:59:30.125 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-11T07:59:30.125 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction Summary 2024-09-11T07:59:30.125 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-09-11T07:59:30.125 INFO:teuthology.orchestra.run.smithi135.stdout:Install 1 Package 2024-09-11T07:59:30.125 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-11T07:59:30.126 INFO:teuthology.orchestra.run.smithi135.stdout:Total download size: 83 k 2024-09-11T07:59:30.126 INFO:teuthology.orchestra.run.smithi135.stdout:Installed size: 375 k 2024-09-11T07:59:30.126 INFO:teuthology.orchestra.run.smithi135.stdout:Downloading Packages: 2024-09-11T07:59:30.323 INFO:teuthology.orchestra.run.smithi135.stdout:cephadm-17.2.7-1623.gd0e6828a.el9.noarch.rpm 419 kB/s | 83 kB 00:00 2024-09-11T07:59:30.324 INFO:teuthology.orchestra.run.smithi135.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:59:30.324 INFO:teuthology.orchestra.run.smithi135.stdout:Total 416 kB/s | 83 kB 00:00 2024-09-11T07:59:30.324 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction check 2024-09-11T07:59:30.329 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction check succeeded. 2024-09-11T07:59:30.329 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction test 2024-09-11T07:59:30.331 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction test succeeded. 2024-09-11T07:59:30.332 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction 2024-09-11T07:59:30.412 INFO:teuthology.orchestra.run.smithi135.stdout: Preparing : 1/1 2024-09-11T07:59:30.813 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 1/1 2024-09-11T07:59:30.901 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 1/1 2024-09-11T07:59:30.993 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-1623.gd0e6828a. 1/1 2024-09-11T07:59:30.993 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T07:59:30.993 INFO:teuthology.orchestra.run.smithi122.stdout:Installed: 2024-09-11T07:59:30.993 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-immutable-object-cache-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:59:30.993 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T07:59:30.993 INFO:teuthology.orchestra.run.smithi122.stdout:Complete! 2024-09-11T07:59:31.204 DEBUG:teuthology.orchestra.run.smithi122:> sudo yum -y install ceph-mgr 2024-09-11T07:59:31.247 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 1/1 2024-09-11T07:59:31.517 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 1/1 2024-09-11T07:59:31.517 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-11T07:59:31.518 INFO:teuthology.orchestra.run.smithi135.stdout:Installed: 2024-09-11T07:59:31.518 INFO:teuthology.orchestra.run.smithi135.stdout: cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T07:59:31.518 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-11T07:59:31.518 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2024-09-11T07:59:31.737 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install ceph-immutable-object-cache 2024-09-11T07:59:31.772 INFO:teuthology.orchestra.run.smithi122.stdout:Last metadata expiration check: 0:00:54 ago on Wed 11 Sep 2024 07:58:37 AM UTC. 2024-09-11T07:59:31.859 INFO:teuthology.orchestra.run.smithi122.stdout:Package ceph-mgr-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T07:59:31.905 INFO:teuthology.orchestra.run.smithi122.stdout:Dependencies resolved. 2024-09-11T07:59:31.906 INFO:teuthology.orchestra.run.smithi122.stdout:Nothing to do. 2024-09-11T07:59:31.906 INFO:teuthology.orchestra.run.smithi122.stdout:Complete! 2024-09-11T07:59:31.980 DEBUG:teuthology.orchestra.run.smithi122:> sudo yum -y install ceph-mgr-dashboard 2024-09-11T07:59:32.322 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:00:54 ago on Wed 11 Sep 2024 07:58:38 AM UTC. 2024-09-11T07:59:32.459 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2024-09-11T07:59:32.460 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-09-11T07:59:32.460 INFO:teuthology.orchestra.run.smithi135.stdout: Package Arch Version Repo Size 2024-09-11T07:59:32.460 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-09-11T07:59:32.460 INFO:teuthology.orchestra.run.smithi135.stdout:Installing: 2024-09-11T07:59:32.460 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 137 k 2024-09-11T07:59:32.461 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-11T07:59:32.461 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction Summary 2024-09-11T07:59:32.461 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-09-11T07:59:32.461 INFO:teuthology.orchestra.run.smithi135.stdout:Install 1 Package 2024-09-11T07:59:32.461 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-11T07:59:32.461 INFO:teuthology.orchestra.run.smithi135.stdout:Total download size: 137 k 2024-09-11T07:59:32.461 INFO:teuthology.orchestra.run.smithi135.stdout:Installed size: 379 k 2024-09-11T07:59:32.461 INFO:teuthology.orchestra.run.smithi135.stdout:Downloading Packages: 2024-09-11T07:59:32.534 INFO:teuthology.orchestra.run.smithi122.stdout:Last metadata expiration check: 0:00:55 ago on Wed 11 Sep 2024 07:58:37 AM UTC. 2024-09-11T07:59:32.663 INFO:teuthology.orchestra.run.smithi135.stdout:ceph-immutable-object-cache-17.2.7-1623.gd0e682 683 kB/s | 137 kB 00:00 2024-09-11T07:59:32.663 INFO:teuthology.orchestra.run.smithi135.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:59:32.663 INFO:teuthology.orchestra.run.smithi135.stdout:Total 677 kB/s | 137 kB 00:00 2024-09-11T07:59:32.664 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction check 2024-09-11T07:59:32.665 INFO:teuthology.orchestra.run.smithi122.stdout:Dependencies resolved. 2024-09-11T07:59:32.666 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-11T07:59:32.666 INFO:teuthology.orchestra.run.smithi122.stdout: Package Arch Version Repository Size 2024-09-11T07:59:32.666 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-11T07:59:32.666 INFO:teuthology.orchestra.run.smithi122.stdout:Installing: 2024-09-11T07:59:32.666 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-dashboard noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 3.4 M 2024-09-11T07:59:32.666 INFO:teuthology.orchestra.run.smithi122.stdout:Installing dependencies: 2024-09-11T07:59:32.667 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-grafana-dashboards noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 24 k 2024-09-11T07:59:32.667 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-prometheus-alerts noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 15 k 2024-09-11T07:59:32.667 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-09-11T07:59:32.667 INFO:teuthology.orchestra.run.smithi122.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-11T07:59:32.667 INFO:teuthology.orchestra.run.smithi122.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-11T07:59:32.667 INFO:teuthology.orchestra.run.smithi122.stdout:Installing weak dependencies: 2024-09-11T07:59:32.667 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-09-11T07:59:32.667 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T07:59:32.667 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction Summary 2024-09-11T07:59:32.667 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-11T07:59:32.667 INFO:teuthology.orchestra.run.smithi122.stdout:Install 7 Packages 2024-09-11T07:59:32.668 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T07:59:32.668 INFO:teuthology.orchestra.run.smithi122.stdout:Total download size: 3.7 M 2024-09-11T07:59:32.668 INFO:teuthology.orchestra.run.smithi122.stdout:Installed size: 80 M 2024-09-11T07:59:32.668 INFO:teuthology.orchestra.run.smithi122.stdout:Downloading Packages: 2024-09-11T07:59:32.671 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction check succeeded. 2024-09-11T07:59:32.671 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction test 2024-09-11T07:59:32.727 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction test succeeded. 2024-09-11T07:59:32.728 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction 2024-09-11T07:59:32.870 INFO:teuthology.orchestra.run.smithi135.stdout: Preparing : 1/1 2024-09-11T07:59:32.890 INFO:teuthology.orchestra.run.smithi122.stdout:(1/7): ceph-grafana-dashboards-17.2.7-1623.gd0e 152 kB/s | 24 kB 00:00 2024-09-11T07:59:32.915 INFO:teuthology.orchestra.run.smithi122.stdout:(2/7): ceph-prometheus-alerts-17.2.7-1623.gd0e6 79 kB/s | 15 kB 00:00 2024-09-11T07:59:32.955 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-1623.gd0e6828a. 1/1 2024-09-11T07:59:32.957 INFO:teuthology.orchestra.run.smithi122.stdout:(3/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 135 kB/s | 9.0 kB 00:00 2024-09-11T07:59:32.981 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-1623.gd0e6828a. 1/1 2024-09-11T07:59:32.981 INFO:teuthology.orchestra.run.smithi135.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T07:59:32.982 INFO:teuthology.orchestra.run.smithi135.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-11T07:59:32.982 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-11T07:59:32.991 INFO:teuthology.orchestra.run.smithi122.stdout:(4/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 969 kB/s | 31 kB 00:00 2024-09-11T07:59:33.016 INFO:teuthology.orchestra.run.smithi122.stdout:(5/7): python3-jwt-2.4.0-1.el9.noarch.rpm 407 kB/s | 41 kB 00:00 2024-09-11T07:59:33.041 INFO:teuthology.orchestra.run.smithi122.stdout:(6/7): python3-routes-2.5.1-5.el9.noarch.rpm 3.7 MB/s | 188 kB 00:00 2024-09-11T07:59:33.158 INFO:teuthology.orchestra.run.smithi122.stdout:(7/7): ceph-mgr-dashboard-17.2.7-1623.gd0e6828a 7.8 MB/s | 3.4 MB 00:00 2024-09-11T07:59:33.161 INFO:teuthology.orchestra.run.smithi122.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:59:33.161 INFO:teuthology.orchestra.run.smithi122.stdout:Total 7.4 MB/s | 3.7 MB 00:00 2024-09-11T07:59:33.162 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction check 2024-09-11T07:59:33.170 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction check succeeded. 2024-09-11T07:59:33.170 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction test 2024-09-11T07:59:33.253 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction test succeeded. 2024-09-11T07:59:33.254 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction 2024-09-11T07:59:33.556 INFO:teuthology.orchestra.run.smithi122.stdout: Preparing : 1/1 2024-09-11T07:59:33.612 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-09-11T07:59:33.643 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-09-11T07:59:33.706 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-09-11T07:59:33.747 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-09-11T07:59:33.775 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-1623.gd0e6828a.el9.n 5/7 2024-09-11T07:59:33.939 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-1623.gd0e6828a. 1/1 2024-09-11T07:59:33.939 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-11T07:59:33.939 INFO:teuthology.orchestra.run.smithi135.stdout:Installed: 2024-09-11T07:59:33.939 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-immutable-object-cache-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:59:33.939 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-11T07:59:33.939 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2024-09-11T07:59:34.153 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install ceph-mgr 2024-09-11T07:59:34.733 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:00:56 ago on Wed 11 Sep 2024 07:58:38 AM UTC. 2024-09-11T07:59:34.758 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-1623.gd0e6828a.el9. 6/7 2024-09-11T07:59:34.788 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-1623.gd0e6828a.el9.noarc 7/7 2024-09-11T07:59:34.828 INFO:teuthology.orchestra.run.smithi135.stdout:Package ceph-mgr-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T07:59:34.870 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2024-09-11T07:59:34.871 INFO:teuthology.orchestra.run.smithi135.stdout:Nothing to do. 2024-09-11T07:59:34.871 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2024-09-11T07:59:34.946 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install ceph-mgr-dashboard 2024-09-11T07:59:35.518 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:00:57 ago on Wed 11 Sep 2024 07:58:38 AM UTC. 2024-09-11T07:59:35.652 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2024-09-11T07:59:35.653 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-09-11T07:59:35.653 INFO:teuthology.orchestra.run.smithi135.stdout: Package Arch Version Repository Size 2024-09-11T07:59:35.653 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-09-11T07:59:35.653 INFO:teuthology.orchestra.run.smithi135.stdout:Installing: 2024-09-11T07:59:35.654 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-dashboard noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 3.4 M 2024-09-11T07:59:35.654 INFO:teuthology.orchestra.run.smithi135.stdout:Installing dependencies: 2024-09-11T07:59:35.654 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-grafana-dashboards noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 24 k 2024-09-11T07:59:35.654 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-prometheus-alerts noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 15 k 2024-09-11T07:59:35.654 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-09-11T07:59:35.654 INFO:teuthology.orchestra.run.smithi135.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-11T07:59:35.654 INFO:teuthology.orchestra.run.smithi135.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-11T07:59:35.654 INFO:teuthology.orchestra.run.smithi135.stdout:Installing weak dependencies: 2024-09-11T07:59:35.654 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-09-11T07:59:35.654 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-11T07:59:35.654 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction Summary 2024-09-11T07:59:35.654 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-09-11T07:59:35.655 INFO:teuthology.orchestra.run.smithi135.stdout:Install 7 Packages 2024-09-11T07:59:35.655 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-11T07:59:35.655 INFO:teuthology.orchestra.run.smithi135.stdout:Total download size: 3.7 M 2024-09-11T07:59:35.655 INFO:teuthology.orchestra.run.smithi135.stdout:Installed size: 80 M 2024-09-11T07:59:35.655 INFO:teuthology.orchestra.run.smithi135.stdout:Downloading Packages: 2024-09-11T07:59:35.850 INFO:teuthology.orchestra.run.smithi135.stdout:(1/7): ceph-grafana-dashboards-17.2.7-1623.gd0e 184 kB/s | 24 kB 00:00 2024-09-11T07:59:35.866 INFO:teuthology.orchestra.run.smithi135.stdout:(2/7): ceph-prometheus-alerts-17.2.7-1623.gd0e6 98 kB/s | 15 kB 00:00 2024-09-11T07:59:35.881 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-1623.gd0e6828a.el9.noarc 7/7 2024-09-11T07:59:35.881 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-1623.gd0e6828a.el9. 1/7 2024-09-11T07:59:35.881 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-1623.gd0e6828a.el9.noarc 2/7 2024-09-11T07:59:35.882 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-1623.gd0e6828a.el9.n 3/7 2024-09-11T07:59:35.882 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-09-11T07:59:35.882 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-09-11T07:59:35.882 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-09-11T07:59:35.908 INFO:teuthology.orchestra.run.smithi135.stdout:(3/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 155 kB/s | 9.0 kB 00:00 2024-09-11T07:59:35.925 INFO:teuthology.orchestra.run.smithi135.stdout:(4/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 2.0 MB/s | 31 kB 00:00 2024-09-11T07:59:35.942 INFO:teuthology.orchestra.run.smithi135.stdout:(5/7): python3-jwt-2.4.0-1.el9.noarch.rpm 541 kB/s | 41 kB 00:00 2024-09-11T07:59:35.976 INFO:teuthology.orchestra.run.smithi135.stdout:(6/7): python3-routes-2.5.1-5.el9.noarch.rpm 3.7 MB/s | 188 kB 00:00 2024-09-11T07:59:36.076 INFO:teuthology.orchestra.run.smithi135.stdout:(7/7): ceph-mgr-dashboard-17.2.7-1623.gd0e6828a 9.3 MB/s | 3.4 MB 00:00 2024-09-11T07:59:36.080 INFO:teuthology.orchestra.run.smithi135.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:59:36.080 INFO:teuthology.orchestra.run.smithi135.stdout:Total 8.6 MB/s | 3.7 MB 00:00 2024-09-11T07:59:36.081 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction check 2024-09-11T07:59:36.089 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction check succeeded. 2024-09-11T07:59:36.089 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction test 2024-09-11T07:59:36.152 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-09-11T07:59:36.152 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T07:59:36.152 INFO:teuthology.orchestra.run.smithi122.stdout:Installed: 2024-09-11T07:59:36.152 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-grafana-dashboards-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T07:59:36.153 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-dashboard-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T07:59:36.153 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-prometheus-alerts-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T07:59:36.153 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-09-11T07:59:36.153 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-09-11T07:59:36.153 INFO:teuthology.orchestra.run.smithi122.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-11T07:59:36.153 INFO:teuthology.orchestra.run.smithi122.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-11T07:59:36.153 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T07:59:36.153 INFO:teuthology.orchestra.run.smithi122.stdout:Complete! 2024-09-11T07:59:36.175 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction test succeeded. 2024-09-11T07:59:36.175 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction 2024-09-11T07:59:36.388 DEBUG:teuthology.orchestra.run.smithi122:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-11T07:59:36.484 INFO:teuthology.orchestra.run.smithi135.stdout: Preparing : 1/1 2024-09-11T07:59:36.552 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-09-11T07:59:36.587 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-09-11T07:59:36.642 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-09-11T07:59:36.682 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-09-11T07:59:36.711 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-1623.gd0e6828a.el9.n 5/7 2024-09-11T07:59:36.948 INFO:teuthology.orchestra.run.smithi122.stdout:Last metadata expiration check: 0:00:59 ago on Wed 11 Sep 2024 07:58:37 AM UTC. 2024-09-11T07:59:37.080 INFO:teuthology.orchestra.run.smithi122.stdout:Dependencies resolved. 2024-09-11T07:59:37.084 INFO:teuthology.orchestra.run.smithi122.stdout:======================================================================================= 2024-09-11T07:59:37.085 INFO:teuthology.orchestra.run.smithi122.stdout: Package Arch Version Repository Size 2024-09-11T07:59:37.085 INFO:teuthology.orchestra.run.smithi122.stdout:======================================================================================= 2024-09-11T07:59:37.085 INFO:teuthology.orchestra.run.smithi122.stdout:Installing: 2024-09-11T07:59:37.085 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 7.4 M 2024-09-11T07:59:37.085 INFO:teuthology.orchestra.run.smithi122.stdout:Installing dependencies: 2024-09-11T07:59:37.085 INFO:teuthology.orchestra.run.smithi122.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-11T07:59:37.086 INFO:teuthology.orchestra.run.smithi122.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-11T07:59:37.086 INFO:teuthology.orchestra.run.smithi122.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-11T07:59:37.086 INFO:teuthology.orchestra.run.smithi122.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-11T07:59:37.086 INFO:teuthology.orchestra.run.smithi122.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-11T07:59:37.086 INFO:teuthology.orchestra.run.smithi122.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-11T07:59:37.086 INFO:teuthology.orchestra.run.smithi122.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-11T07:59:37.086 INFO:teuthology.orchestra.run.smithi122.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-11T07:59:37.086 INFO:teuthology.orchestra.run.smithi122.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-11T07:59:37.086 INFO:teuthology.orchestra.run.smithi122.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-11T07:59:37.086 INFO:teuthology.orchestra.run.smithi122.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-11T07:59:37.086 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T07:59:37.086 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction Summary 2024-09-11T07:59:37.086 INFO:teuthology.orchestra.run.smithi122.stdout:======================================================================================= 2024-09-11T07:59:37.087 INFO:teuthology.orchestra.run.smithi122.stdout:Install 12 Packages 2024-09-11T07:59:37.087 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T07:59:37.087 INFO:teuthology.orchestra.run.smithi122.stdout:Total download size: 38 M 2024-09-11T07:59:37.087 INFO:teuthology.orchestra.run.smithi122.stdout:Installed size: 212 M 2024-09-11T07:59:37.087 INFO:teuthology.orchestra.run.smithi122.stdout:Downloading Packages: 2024-09-11T07:59:37.704 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-1623.gd0e6828a.el9. 6/7 2024-09-11T07:59:37.739 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-1623.gd0e6828a.el9.noarc 7/7 2024-09-11T07:59:37.925 INFO:teuthology.orchestra.run.smithi122.stdout:(1/12): ceph-mgr-diskprediction-local-17.2.7-16 17 MB/s | 7.4 MB 00:00 2024-09-11T07:59:38.033 INFO:teuthology.orchestra.run.smithi122.stdout:(2/12): libquadmath-11.5.0-2.el9.x86_64.rpm 344 kB/s | 191 kB 00:00 2024-09-11T07:59:38.058 INFO:teuthology.orchestra.run.smithi122.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 245 kB/s | 33 kB 00:00 2024-09-11T07:59:38.151 INFO:teuthology.orchestra.run.smithi122.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 198 kB/s | 18 kB 00:00 2024-09-11T07:59:38.226 INFO:teuthology.orchestra.run.smithi122.stdout:(5/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 16 MB/s | 3.0 MB 00:00 2024-09-11T07:59:38.251 INFO:teuthology.orchestra.run.smithi122.stdout:(6/12): openblas-0.3.26-2.el9.x86_64.rpm 384 kB/s | 38 kB 00:00 2024-09-11T07:59:38.385 INFO:teuthology.orchestra.run.smithi122.stdout:(7/12): openblas-openmp-0.3.26-2.el9.x86_64.rpm 31 MB/s | 4.9 MB 00:00 2024-09-11T07:59:38.410 INFO:teuthology.orchestra.run.smithi122.stdout:(8/12): python3-devel-3.9.19-8.el9.x86_64.rpm 1.5 MB/s | 245 kB 00:00 2024-09-11T07:59:38.519 INFO:teuthology.orchestra.run.smithi122.stdout:(9/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 38 MB/s | 5.1 MB 00:00 2024-09-11T07:59:38.544 INFO:teuthology.orchestra.run.smithi122.stdout:(10/12): libgfortran-11.5.0-2.el9.x86_64.rpm 751 kB/s | 801 kB 00:01 2024-09-11T07:59:38.569 INFO:teuthology.orchestra.run.smithi122.stdout:(11/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 1.5 MB/s | 250 kB 00:00 2024-09-11T07:59:38.803 INFO:teuthology.orchestra.run.smithi122.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 57 MB/s | 16 MB 00:00 2024-09-11T07:59:38.803 INFO:teuthology.orchestra.run.smithi122.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:59:38.804 INFO:teuthology.orchestra.run.smithi122.stdout:Total 22 MB/s | 38 MB 00:01 2024-09-11T07:59:38.824 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-1623.gd0e6828a.el9.noarc 7/7 2024-09-11T07:59:38.825 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-1623.gd0e6828a.el9. 1/7 2024-09-11T07:59:38.825 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-1623.gd0e6828a.el9.noarc 2/7 2024-09-11T07:59:38.825 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-1623.gd0e6828a.el9.n 3/7 2024-09-11T07:59:38.825 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-09-11T07:59:38.825 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-09-11T07:59:38.825 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-09-11T07:59:39.108 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-09-11T07:59:39.108 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-11T07:59:39.108 INFO:teuthology.orchestra.run.smithi135.stdout:Installed: 2024-09-11T07:59:39.108 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-grafana-dashboards-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T07:59:39.109 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-dashboard-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T07:59:39.109 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-prometheus-alerts-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T07:59:39.109 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-09-11T07:59:39.109 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-09-11T07:59:39.109 INFO:teuthology.orchestra.run.smithi135.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-11T07:59:39.109 INFO:teuthology.orchestra.run.smithi135.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-11T07:59:39.109 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-11T07:59:39.109 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2024-09-11T07:59:39.160 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction check 2024-09-11T07:59:39.186 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction check succeeded. 2024-09-11T07:59:39.187 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction test 2024-09-11T07:59:39.337 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-11T07:59:39.692 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction test succeeded. 2024-09-11T07:59:39.693 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction 2024-09-11T07:59:39.905 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:01:01 ago on Wed 11 Sep 2024 07:58:38 AM UTC. 2024-09-11T07:59:40.038 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2024-09-11T07:59:40.039 INFO:teuthology.orchestra.run.smithi135.stdout:======================================================================================= 2024-09-11T07:59:40.040 INFO:teuthology.orchestra.run.smithi135.stdout: Package Arch Version Repository Size 2024-09-11T07:59:40.040 INFO:teuthology.orchestra.run.smithi135.stdout:======================================================================================= 2024-09-11T07:59:40.040 INFO:teuthology.orchestra.run.smithi135.stdout:Installing: 2024-09-11T07:59:40.040 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 7.4 M 2024-09-11T07:59:40.040 INFO:teuthology.orchestra.run.smithi135.stdout:Installing dependencies: 2024-09-11T07:59:40.040 INFO:teuthology.orchestra.run.smithi135.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-11T07:59:40.040 INFO:teuthology.orchestra.run.smithi135.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-11T07:59:40.040 INFO:teuthology.orchestra.run.smithi135.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-11T07:59:40.040 INFO:teuthology.orchestra.run.smithi135.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-11T07:59:40.040 INFO:teuthology.orchestra.run.smithi135.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-11T07:59:40.040 INFO:teuthology.orchestra.run.smithi135.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-11T07:59:40.040 INFO:teuthology.orchestra.run.smithi135.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-11T07:59:40.041 INFO:teuthology.orchestra.run.smithi135.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-11T07:59:40.041 INFO:teuthology.orchestra.run.smithi135.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-11T07:59:40.041 INFO:teuthology.orchestra.run.smithi135.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-11T07:59:40.041 INFO:teuthology.orchestra.run.smithi135.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-11T07:59:40.041 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-11T07:59:40.041 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction Summary 2024-09-11T07:59:40.041 INFO:teuthology.orchestra.run.smithi135.stdout:======================================================================================= 2024-09-11T07:59:40.041 INFO:teuthology.orchestra.run.smithi135.stdout:Install 12 Packages 2024-09-11T07:59:40.041 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-11T07:59:40.042 INFO:teuthology.orchestra.run.smithi135.stdout:Total download size: 38 M 2024-09-11T07:59:40.042 INFO:teuthology.orchestra.run.smithi135.stdout:Installed size: 212 M 2024-09-11T07:59:40.042 INFO:teuthology.orchestra.run.smithi135.stdout:Downloading Packages: 2024-09-11T07:59:40.455 INFO:teuthology.orchestra.run.smithi122.stdout: Preparing : 1/1 2024-09-11T07:59:40.535 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-11T07:59:40.576 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-11T07:59:40.682 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-11T07:59:40.727 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-11T07:59:40.734 INFO:teuthology.orchestra.run.smithi135.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 1.1 MB/s | 191 kB 00:00 2024-09-11T07:59:40.776 INFO:teuthology.orchestra.run.smithi135.stdout:(2/12): libgfortran-11.5.0-2.el9.x86_64.rpm 3.7 MB/s | 801 kB 00:00 2024-09-11T07:59:40.911 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-11T07:59:40.943 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-11T07:59:41.029 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-11T07:59:41.060 INFO:teuthology.orchestra.run.smithi135.stdout:(3/12): ceph-mgr-diskprediction-local-17.2.7-16 15 MB/s | 7.4 MB 00:00 2024-09-11T07:59:41.085 INFO:teuthology.orchestra.run.smithi135.stdout:(4/12): flexiblas-3.0.4-8.el9.x86_64.rpm 93 kB/s | 33 kB 00:00 2024-09-11T07:59:41.285 INFO:teuthology.orchestra.run.smithi135.stdout:(5/12): openblas-0.3.26-2.el9.x86_64.rpm 192 kB/s | 38 kB 00:00 2024-09-11T07:59:41.344 INFO:teuthology.orchestra.run.smithi135.stdout:(6/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 64 kB/s | 18 kB 00:00 2024-09-11T07:59:41.677 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-11T07:59:41.788 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-11T07:59:41.811 INFO:teuthology.orchestra.run.smithi135.stdout:(7/12): python3-devel-3.9.19-8.el9.x86_64.rpm 524 kB/s | 245 kB 00:00 2024-09-11T07:59:41.978 INFO:teuthology.orchestra.run.smithi135.stdout:(8/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 2.5 MB/s | 3.0 MB 00:01 2024-09-11T07:59:42.095 INFO:teuthology.orchestra.run.smithi135.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.1 MB/s | 250 kB 00:00 2024-09-11T07:59:43.029 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-11T07:59:43.520 INFO:teuthology.orchestra.run.smithi135.stdout:(10/12): openblas-openmp-0.3.26-2.el9.x86_64.rp 2.2 MB/s | 4.9 MB 00:02 2024-09-11T07:59:44.064 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-11T07:59:44.101 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-1623.gd0e68 12/12 2024-09-11T07:59:44.561 INFO:teuthology.orchestra.run.smithi135.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 1.9 MB/s | 5.1 MB 00:02 2024-09-11T07:59:46.675 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-1623.gd0e68 12/12 2024-09-11T07:59:46.676 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-1623.gd0e68 1/12 2024-09-11T07:59:46.676 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-11T07:59:46.676 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-11T07:59:46.676 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-11T07:59:46.676 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-11T07:59:46.676 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-11T07:59:46.676 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-11T07:59:46.676 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-11T07:59:46.676 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-11T07:59:46.676 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-11T07:59:46.676 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-11T07:59:47.195 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-11T07:59:47.195 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T07:59:47.195 INFO:teuthology.orchestra.run.smithi122.stdout:Installed: 2024-09-11T07:59:47.195 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-diskprediction-local-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T07:59:47.195 INFO:teuthology.orchestra.run.smithi122.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-11T07:59:47.195 INFO:teuthology.orchestra.run.smithi122.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-11T07:59:47.195 INFO:teuthology.orchestra.run.smithi122.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-11T07:59:47.195 INFO:teuthology.orchestra.run.smithi122.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-11T07:59:47.195 INFO:teuthology.orchestra.run.smithi122.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-11T07:59:47.195 INFO:teuthology.orchestra.run.smithi122.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-11T07:59:47.196 INFO:teuthology.orchestra.run.smithi122.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-11T07:59:47.196 INFO:teuthology.orchestra.run.smithi122.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-11T07:59:47.196 INFO:teuthology.orchestra.run.smithi122.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-11T07:59:47.196 INFO:teuthology.orchestra.run.smithi122.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-11T07:59:47.196 INFO:teuthology.orchestra.run.smithi122.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-11T07:59:47.196 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T07:59:47.196 INFO:teuthology.orchestra.run.smithi122.stdout:Complete! 2024-09-11T07:59:47.700 DEBUG:teuthology.orchestra.run.smithi122:> sudo yum -y install ceph-mgr-rook 2024-09-11T07:59:48.279 INFO:teuthology.orchestra.run.smithi122.stdout:Last metadata expiration check: 0:01:11 ago on Wed 11 Sep 2024 07:58:37 AM UTC. 2024-09-11T07:59:48.411 INFO:teuthology.orchestra.run.smithi122.stdout:Dependencies resolved. 2024-09-11T07:59:48.412 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-11T07:59:48.412 INFO:teuthology.orchestra.run.smithi122.stdout: Package Arch Version Repository Size 2024-09-11T07:59:48.412 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-11T07:59:48.412 INFO:teuthology.orchestra.run.smithi122.stdout:Installing: 2024-09-11T07:59:48.412 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-rook noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 47 k 2024-09-11T07:59:48.412 INFO:teuthology.orchestra.run.smithi122.stdout:Installing dependencies: 2024-09-11T07:59:48.413 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-11T07:59:48.413 INFO:teuthology.orchestra.run.smithi122.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-11T07:59:48.413 INFO:teuthology.orchestra.run.smithi122.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-11T07:59:48.413 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-11T07:59:48.413 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-11T07:59:48.413 INFO:teuthology.orchestra.run.smithi122.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-11T07:59:48.413 INFO:teuthology.orchestra.run.smithi122.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-11T07:59:48.413 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-11T07:59:48.413 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-11T07:59:48.413 INFO:teuthology.orchestra.run.smithi122.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-11T07:59:48.413 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-11T07:59:48.413 INFO:teuthology.orchestra.run.smithi122.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-11T07:59:48.413 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T07:59:48.414 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction Summary 2024-09-11T07:59:48.414 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-11T07:59:48.414 INFO:teuthology.orchestra.run.smithi122.stdout:Install 13 Packages 2024-09-11T07:59:48.414 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T07:59:48.414 INFO:teuthology.orchestra.run.smithi122.stdout:Total download size: 2.2 M 2024-09-11T07:59:48.414 INFO:teuthology.orchestra.run.smithi122.stdout:Installed size: 26 M 2024-09-11T07:59:48.415 INFO:teuthology.orchestra.run.smithi122.stdout:Downloading Packages: 2024-09-11T07:59:48.818 INFO:teuthology.orchestra.run.smithi122.stdout:(1/13): python3-jsonpointer-2.0-4.el9.noarch.rp 172 kB/s | 19 kB 00:00 2024-09-11T07:59:48.843 INFO:teuthology.orchestra.run.smithi122.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 188 kB/s | 26 kB 00:00 2024-09-11T07:59:48.902 INFO:teuthology.orchestra.run.smithi122.stdout:(3/13): ceph-mgr-rook-17.2.7-1623.gd0e6828a.el9 243 kB/s | 47 kB 00:00 2024-09-11T07:59:48.952 INFO:teuthology.orchestra.run.smithi122.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.4 MB/s | 159 kB 00:00 2024-09-11T07:59:48.978 INFO:teuthology.orchestra.run.smithi122.stdout:(5/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.4 MB/s | 222 kB 00:00 2024-09-11T07:59:49.019 INFO:teuthology.orchestra.run.smithi122.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 801 kB/s | 54 kB 00:00 2024-09-11T07:59:49.045 INFO:teuthology.orchestra.run.smithi122.stdout:(7/13): python3-cachetools-4.2.4-1.el9.noarch.r 480 kB/s | 32 kB 00:00 2024-09-11T07:59:49.078 INFO:teuthology.orchestra.run.smithi122.stdout:(8/13): python3-google-auth-2.34.0-1.el9.noarch 8.1 MB/s | 220 kB 00:00 2024-09-11T07:59:49.104 INFO:teuthology.orchestra.run.smithi122.stdout:(9/13): python3-certifi-2023.05.07-4.el9.noarch 168 kB/s | 14 kB 00:00 2024-09-11T07:59:49.129 INFO:teuthology.orchestra.run.smithi122.stdout:(10/13): python3-pyasn1-modules-0.4.8-6.el9.noa 1.2 MB/s | 279 kB 00:00 2024-09-11T07:59:49.154 INFO:teuthology.orchestra.run.smithi122.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-09-11T07:59:49.196 INFO:teuthology.orchestra.run.smithi122.stdout:(12/13): python3-kubernetes-26.1.0-2.el9.noarch 8.8 MB/s | 1.0 MB 00:00 2024-09-11T07:59:49.221 INFO:teuthology.orchestra.run.smithi122.stdout:(13/13): python3-websocket-client-1.2.3-2.el9.n 972 kB/s | 90 kB 00:00 2024-09-11T07:59:49.224 INFO:teuthology.orchestra.run.smithi122.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:59:49.224 INFO:teuthology.orchestra.run.smithi122.stdout:Total 2.7 MB/s | 2.2 MB 00:00 2024-09-11T07:59:49.288 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction check 2024-09-11T07:59:49.303 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction check succeeded. 2024-09-11T07:59:49.303 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction test 2024-09-11T07:59:49.392 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction test succeeded. 2024-09-11T07:59:49.393 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction 2024-09-11T07:59:49.900 INFO:teuthology.orchestra.run.smithi122.stdout: Preparing : 1/1 2024-09-11T07:59:50.076 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-11T07:59:50.131 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-11T07:59:50.187 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-11T07:59:50.224 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-11T07:59:50.265 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-11T07:59:50.375 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-11T07:59:50.485 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-11T07:59:50.533 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-11T07:59:51.246 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-11T07:59:51.306 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-11T07:59:51.340 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-11T07:59:51.385 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-11T07:59:51.409 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mgr-rook-2:17.2.7-1623.gd0e6828a.el9.noarch 13/13 2024-09-11T07:59:52.165 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-1623.gd0e6828a.el9.noarch 13/13 2024-09-11T07:59:52.165 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mgr-rook-2:17.2.7-1623.gd0e6828a.el9.noarch 1/13 2024-09-11T07:59:52.165 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-11T07:59:52.165 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-11T07:59:52.165 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-11T07:59:52.165 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-11T07:59:52.165 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-11T07:59:52.166 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-11T07:59:52.166 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-11T07:59:52.166 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-11T07:59:52.166 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-11T07:59:52.166 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-11T07:59:52.166 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-11T07:59:52.509 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-11T07:59:52.510 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T07:59:52.510 INFO:teuthology.orchestra.run.smithi122.stdout:Installed: 2024-09-11T07:59:52.510 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-rook-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T07:59:52.510 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-11T07:59:52.510 INFO:teuthology.orchestra.run.smithi122.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-11T07:59:52.510 INFO:teuthology.orchestra.run.smithi122.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-11T07:59:52.510 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-11T07:59:52.510 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-11T07:59:52.510 INFO:teuthology.orchestra.run.smithi122.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-11T07:59:52.510 INFO:teuthology.orchestra.run.smithi122.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-11T07:59:52.510 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-11T07:59:52.510 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-11T07:59:52.511 INFO:teuthology.orchestra.run.smithi122.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-11T07:59:52.511 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-11T07:59:52.511 INFO:teuthology.orchestra.run.smithi122.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-11T07:59:52.511 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T07:59:52.511 INFO:teuthology.orchestra.run.smithi122.stdout:Complete! 2024-09-11T07:59:52.771 DEBUG:teuthology.orchestra.run.smithi122:> sudo yum -y install ceph-mgr-cephadm 2024-09-11T07:59:53.350 INFO:teuthology.orchestra.run.smithi122.stdout:Last metadata expiration check: 0:01:16 ago on Wed 11 Sep 2024 07:58:37 AM UTC. 2024-09-11T07:59:53.484 INFO:teuthology.orchestra.run.smithi122.stdout:Dependencies resolved. 2024-09-11T07:59:53.485 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-11T07:59:53.485 INFO:teuthology.orchestra.run.smithi122.stdout: Package Arch Version Repository Size 2024-09-11T07:59:53.485 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-11T07:59:53.485 INFO:teuthology.orchestra.run.smithi122.stdout:Installing: 2024-09-11T07:59:53.485 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-cephadm noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 124 k 2024-09-11T07:59:53.485 INFO:teuthology.orchestra.run.smithi122.stdout:Installing dependencies: 2024-09-11T07:59:53.485 INFO:teuthology.orchestra.run.smithi122.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-11T07:59:53.485 INFO:teuthology.orchestra.run.smithi122.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-11T07:59:53.486 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-11T07:59:53.486 INFO:teuthology.orchestra.run.smithi122.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-11T07:59:53.486 INFO:teuthology.orchestra.run.smithi122.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-11T07:59:53.486 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T07:59:53.486 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction Summary 2024-09-11T07:59:53.486 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-11T07:59:53.486 INFO:teuthology.orchestra.run.smithi122.stdout:Install 6 Packages 2024-09-11T07:59:53.486 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T07:59:53.487 INFO:teuthology.orchestra.run.smithi122.stdout:Total download size: 7.0 M 2024-09-11T07:59:53.487 INFO:teuthology.orchestra.run.smithi122.stdout:Installed size: 33 M 2024-09-11T07:59:53.487 INFO:teuthology.orchestra.run.smithi122.stdout:Downloading Packages: 2024-09-11T07:59:54.032 INFO:teuthology.orchestra.run.smithi122.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.2 MB/s | 248 kB 00:00 2024-09-11T07:59:54.058 INFO:teuthology.orchestra.run.smithi122.stdout:(2/6): ceph-mgr-cephadm-17.2.7-1623.gd0e6828a.e 526 kB/s | 124 kB 00:00 2024-09-11T07:59:54.116 INFO:teuthology.orchestra.run.smithi122.stdout:(3/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 6.4 MB/s | 548 kB 00:00 2024-09-11T07:59:54.141 INFO:teuthology.orchestra.run.smithi122.stdout:(4/6): python3-typing-extensions-4.12.2-2.el9.n 3.2 MB/s | 74 kB 00:00 2024-09-11T07:59:54.167 INFO:teuthology.orchestra.run.smithi122.stdout:(5/6): python3-natsort-7.1.1-5.el9.noarch.rpm 529 kB/s | 58 kB 00:00 2024-09-11T07:59:54.967 INFO:teuthology.orchestra.run.smithi122.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 5.2 MB/s | 6.0 MB 00:01 2024-09-11T07:59:54.969 INFO:teuthology.orchestra.run.smithi122.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:59:54.969 INFO:teuthology.orchestra.run.smithi122.stdout:Total 4.7 MB/s | 7.0 MB 00:01 2024-09-11T07:59:55.040 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction check 2024-09-11T07:59:55.051 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction check succeeded. 2024-09-11T07:59:55.051 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction test 2024-09-11T07:59:55.150 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction test succeeded. 2024-09-11T07:59:55.151 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction 2024-09-11T07:59:55.461 INFO:teuthology.orchestra.run.smithi122.stdout: Preparing : 1/1 2024-09-11T07:59:55.630 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-11T07:59:55.685 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-11T07:59:56.209 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-11T07:59:56.312 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-11T07:59:56.385 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-11T07:59:56.423 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 6/6 2024-09-11T07:59:57.321 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 6/6 2024-09-11T07:59:57.321 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 1/6 2024-09-11T07:59:57.321 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-11T07:59:57.321 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-11T07:59:57.321 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-11T07:59:57.321 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-11T07:59:57.581 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-11T07:59:57.582 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T07:59:57.582 INFO:teuthology.orchestra.run.smithi122.stdout:Installed: 2024-09-11T07:59:57.582 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T07:59:57.582 INFO:teuthology.orchestra.run.smithi122.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-11T07:59:57.582 INFO:teuthology.orchestra.run.smithi122.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-11T07:59:57.582 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-11T07:59:57.582 INFO:teuthology.orchestra.run.smithi122.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-11T07:59:57.582 INFO:teuthology.orchestra.run.smithi122.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-11T07:59:57.582 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T07:59:57.582 INFO:teuthology.orchestra.run.smithi122.stdout:Complete! 2024-09-11T07:59:57.849 DEBUG:teuthology.orchestra.run.smithi122:> sudo yum -y install ceph-fuse 2024-09-11T07:59:58.415 INFO:teuthology.orchestra.run.smithi122.stdout:Last metadata expiration check: 0:01:21 ago on Wed 11 Sep 2024 07:58:37 AM UTC. 2024-09-11T07:59:58.547 INFO:teuthology.orchestra.run.smithi122.stdout:Dependencies resolved. 2024-09-11T07:59:58.548 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-11T07:59:58.548 INFO:teuthology.orchestra.run.smithi122.stdout: Package Arch Version Repository Size 2024-09-11T07:59:58.548 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-11T07:59:58.548 INFO:teuthology.orchestra.run.smithi122.stdout:Installing: 2024-09-11T07:59:58.548 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-fuse x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 813 k 2024-09-11T07:59:58.548 INFO:teuthology.orchestra.run.smithi122.stdout:Installing dependencies: 2024-09-11T07:59:58.548 INFO:teuthology.orchestra.run.smithi122.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-11T07:59:58.548 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T07:59:58.548 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction Summary 2024-09-11T07:59:58.548 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-11T07:59:58.548 INFO:teuthology.orchestra.run.smithi122.stdout:Install 2 Packages 2024-09-11T07:59:58.549 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T07:59:58.549 INFO:teuthology.orchestra.run.smithi122.stdout:Total download size: 893 k 2024-09-11T07:59:58.549 INFO:teuthology.orchestra.run.smithi122.stdout:Installed size: 2.6 M 2024-09-11T07:59:58.549 INFO:teuthology.orchestra.run.smithi122.stdout:Downloading Packages: 2024-09-11T07:59:58.744 INFO:teuthology.orchestra.run.smithi122.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 548 kB/s | 80 kB 00:00 2024-09-11T07:59:58.878 INFO:teuthology.orchestra.run.smithi122.stdout:(2/2): ceph-fuse-17.2.7-1623.gd0e6828a.el9.x86_ 2.8 MB/s | 813 kB 00:00 2024-09-11T07:59:58.878 INFO:teuthology.orchestra.run.smithi122.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:59:58.878 INFO:teuthology.orchestra.run.smithi122.stdout:Total 2.6 MB/s | 893 kB 00:00 2024-09-11T07:59:58.890 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction check 2024-09-11T07:59:58.911 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction check succeeded. 2024-09-11T07:59:58.911 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction test 2024-09-11T07:59:58.995 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction test succeeded. 2024-09-11T07:59:58.995 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction 2024-09-11T07:59:59.186 INFO:teuthology.orchestra.run.smithi122.stdout: Preparing : 1/1 2024-09-11T07:59:59.333 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-11T07:59:59.410 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/2 2024-09-11T08:00:00.066 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/2 2024-09-11T08:00:00.067 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/2 2024-09-11T08:00:00.314 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-11T08:00:00.314 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T08:00:00.314 INFO:teuthology.orchestra.run.smithi122.stdout:Installed: 2024-09-11T08:00:00.314 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-11T08:00:00.314 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T08:00:00.314 INFO:teuthology.orchestra.run.smithi122.stdout:Complete! 2024-09-11T08:00:00.602 DEBUG:teuthology.orchestra.run.smithi122:> sudo yum -y install ceph-volume 2024-09-11T08:00:01.135 INFO:teuthology.orchestra.run.smithi135.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 874 kB/s | 16 MB 00:19 2024-09-11T08:00:01.136 INFO:teuthology.orchestra.run.smithi135.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:00:01.137 INFO:teuthology.orchestra.run.smithi135.stdout:Total 1.8 MB/s | 38 MB 00:21 2024-09-11T08:00:01.184 INFO:teuthology.orchestra.run.smithi122.stdout:Last metadata expiration check: 0:01:24 ago on Wed 11 Sep 2024 07:58:37 AM UTC. 2024-09-11T08:00:01.322 INFO:teuthology.orchestra.run.smithi122.stdout:Dependencies resolved. 2024-09-11T08:00:01.323 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-11T08:00:01.323 INFO:teuthology.orchestra.run.smithi122.stdout: Package Arch Version Repository Size 2024-09-11T08:00:01.323 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-11T08:00:01.323 INFO:teuthology.orchestra.run.smithi122.stdout:Installing: 2024-09-11T08:00:01.323 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-volume noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 270 k 2024-09-11T08:00:01.323 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T08:00:01.323 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction Summary 2024-09-11T08:00:01.323 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-11T08:00:01.323 INFO:teuthology.orchestra.run.smithi122.stdout:Install 1 Package 2024-09-11T08:00:01.323 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T08:00:01.324 INFO:teuthology.orchestra.run.smithi122.stdout:Total download size: 270 k 2024-09-11T08:00:01.324 INFO:teuthology.orchestra.run.smithi122.stdout:Installed size: 1.3 M 2024-09-11T08:00:01.324 INFO:teuthology.orchestra.run.smithi122.stdout:Downloading Packages: 2024-09-11T08:00:01.501 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction check 2024-09-11T08:00:01.527 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction check succeeded. 2024-09-11T08:00:01.528 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction test 2024-09-11T08:00:01.566 INFO:teuthology.orchestra.run.smithi122.stdout:ceph-volume-17.2.7-1623.gd0e6828a.el9.noarch.rp 1.1 MB/s | 270 kB 00:00 2024-09-11T08:00:01.567 INFO:teuthology.orchestra.run.smithi122.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:00:01.567 INFO:teuthology.orchestra.run.smithi122.stdout:Total 1.1 MB/s | 270 kB 00:00 2024-09-11T08:00:01.567 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction check 2024-09-11T08:00:01.576 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction check succeeded. 2024-09-11T08:00:01.576 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction test 2024-09-11T08:00:01.611 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction test succeeded. 2024-09-11T08:00:01.611 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction 2024-09-11T08:00:01.823 INFO:teuthology.orchestra.run.smithi122.stdout: Preparing : 1/1 2024-09-11T08:00:01.885 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-volume-2:17.2.7-1623.gd0e6828a.el9.noarch 1/1 2024-09-11T08:00:01.901 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-volume-2:17.2.7-1623.gd0e6828a.el9.noarch 1/1 2024-09-11T08:00:01.901 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T08:00:01.902 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-11T08:00:01.902 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T08:00:02.056 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction test succeeded. 2024-09-11T08:00:02.057 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction 2024-09-11T08:00:02.826 INFO:teuthology.orchestra.run.smithi135.stdout: Preparing : 1/1 2024-09-11T08:00:02.906 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-11T08:00:02.949 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-11T08:00:02.951 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-volume-2:17.2.7-1623.gd0e6828a.el9.noarch 1/1 2024-09-11T08:00:02.951 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T08:00:02.951 INFO:teuthology.orchestra.run.smithi122.stdout:Installed: 2024-09-11T08:00:02.951 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-volume-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T08:00:02.951 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T08:00:02.951 INFO:teuthology.orchestra.run.smithi122.stdout:Complete! 2024-09-11T08:00:03.049 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-11T08:00:03.093 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-11T08:00:03.287 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-11T08:00:03.296 DEBUG:teuthology.orchestra.run.smithi122:> sudo yum -y install librados-devel 2024-09-11T08:00:03.317 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-11T08:00:03.394 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-11T08:00:03.875 INFO:teuthology.orchestra.run.smithi122.stdout:Last metadata expiration check: 0:01:26 ago on Wed 11 Sep 2024 07:58:37 AM UTC. 2024-09-11T08:00:04.007 INFO:teuthology.orchestra.run.smithi122.stdout:Dependencies resolved. 2024-09-11T08:00:04.008 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-11T08:00:04.008 INFO:teuthology.orchestra.run.smithi122.stdout: Package Arch Version Repo Size 2024-09-11T08:00:04.008 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-11T08:00:04.009 INFO:teuthology.orchestra.run.smithi122.stdout:Installing: 2024-09-11T08:00:04.009 INFO:teuthology.orchestra.run.smithi122.stdout: librados-devel x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 126 k 2024-09-11T08:00:04.009 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T08:00:04.009 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction Summary 2024-09-11T08:00:04.009 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-11T08:00:04.009 INFO:teuthology.orchestra.run.smithi122.stdout:Install 1 Package 2024-09-11T08:00:04.009 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T08:00:04.009 INFO:teuthology.orchestra.run.smithi122.stdout:Total download size: 126 k 2024-09-11T08:00:04.009 INFO:teuthology.orchestra.run.smithi122.stdout:Installed size: 448 k 2024-09-11T08:00:04.009 INFO:teuthology.orchestra.run.smithi122.stdout:Downloading Packages: 2024-09-11T08:00:04.048 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-11T08:00:04.171 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-11T08:00:04.222 INFO:teuthology.orchestra.run.smithi122.stdout:librados-devel-17.2.7-1623.gd0e6828a.el9.x86_64 593 kB/s | 126 kB 00:00 2024-09-11T08:00:04.223 INFO:teuthology.orchestra.run.smithi122.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:00:04.223 INFO:teuthology.orchestra.run.smithi122.stdout:Total 588 kB/s | 126 kB 00:00 2024-09-11T08:00:04.223 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction check 2024-09-11T08:00:04.229 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction check succeeded. 2024-09-11T08:00:04.229 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction test 2024-09-11T08:00:04.292 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction test succeeded. 2024-09-11T08:00:04.292 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction 2024-09-11T08:00:04.446 INFO:teuthology.orchestra.run.smithi122.stdout: Preparing : 1/1 2024-09-11T08:00:04.552 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : librados-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:00:04.982 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: librados-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:00:05.256 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librados-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:00:05.257 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T08:00:05.257 INFO:teuthology.orchestra.run.smithi122.stdout:Installed: 2024-09-11T08:00:05.257 INFO:teuthology.orchestra.run.smithi122.stdout: librados-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:00:05.257 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T08:00:05.257 INFO:teuthology.orchestra.run.smithi122.stdout:Complete! 2024-09-11T08:00:05.423 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-11T08:00:05.483 DEBUG:teuthology.orchestra.run.smithi122:> sudo yum -y install libcephfs2 2024-09-11T08:00:06.050 INFO:teuthology.orchestra.run.smithi122.stdout:Last metadata expiration check: 0:01:29 ago on Wed 11 Sep 2024 07:58:37 AM UTC. 2024-09-11T08:00:06.141 INFO:teuthology.orchestra.run.smithi122.stdout:Package libcephfs2-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T08:00:06.183 INFO:teuthology.orchestra.run.smithi122.stdout:Dependencies resolved. 2024-09-11T08:00:06.184 INFO:teuthology.orchestra.run.smithi122.stdout:Nothing to do. 2024-09-11T08:00:06.185 INFO:teuthology.orchestra.run.smithi122.stdout:Complete! 2024-09-11T08:00:06.293 DEBUG:teuthology.orchestra.run.smithi122:> sudo yum -y install libcephfs-devel 2024-09-11T08:00:06.466 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-11T08:00:06.491 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-1623.gd0e68 12/12 2024-09-11T08:00:06.872 INFO:teuthology.orchestra.run.smithi122.stdout:Last metadata expiration check: 0:01:29 ago on Wed 11 Sep 2024 07:58:37 AM UTC. 2024-09-11T08:00:07.004 INFO:teuthology.orchestra.run.smithi122.stdout:Dependencies resolved. 2024-09-11T08:00:07.004 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-11T08:00:07.004 INFO:teuthology.orchestra.run.smithi122.stdout: Package Arch Version Repo Size 2024-09-11T08:00:07.004 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-11T08:00:07.004 INFO:teuthology.orchestra.run.smithi122.stdout:Installing: 2024-09-11T08:00:07.004 INFO:teuthology.orchestra.run.smithi122.stdout: libcephfs-devel x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 25 k 2024-09-11T08:00:07.005 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T08:00:07.005 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction Summary 2024-09-11T08:00:07.005 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-11T08:00:07.005 INFO:teuthology.orchestra.run.smithi122.stdout:Install 1 Package 2024-09-11T08:00:07.005 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T08:00:07.005 INFO:teuthology.orchestra.run.smithi122.stdout:Total download size: 25 k 2024-09-11T08:00:07.005 INFO:teuthology.orchestra.run.smithi122.stdout:Installed size: 110 k 2024-09-11T08:00:07.006 INFO:teuthology.orchestra.run.smithi122.stdout:Downloading Packages: 2024-09-11T08:00:07.160 INFO:teuthology.orchestra.run.smithi122.stdout:libcephfs-devel-17.2.7-1623.gd0e6828a.el9.x86_6 161 kB/s | 25 kB 00:00 2024-09-11T08:00:07.161 INFO:teuthology.orchestra.run.smithi122.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:00:07.161 INFO:teuthology.orchestra.run.smithi122.stdout:Total 160 kB/s | 25 kB 00:00 2024-09-11T08:00:07.161 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction check 2024-09-11T08:00:07.165 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction check succeeded. 2024-09-11T08:00:07.165 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction test 2024-09-11T08:00:07.166 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction test succeeded. 2024-09-11T08:00:07.166 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction 2024-09-11T08:00:07.249 INFO:teuthology.orchestra.run.smithi122.stdout: Preparing : 1/1 2024-09-11T08:00:07.374 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libcephfs-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:00:07.712 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:00:07.937 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libcephfs-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:00:07.937 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T08:00:07.937 INFO:teuthology.orchestra.run.smithi122.stdout:Installed: 2024-09-11T08:00:07.937 INFO:teuthology.orchestra.run.smithi122.stdout: libcephfs-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:00:07.937 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T08:00:07.938 INFO:teuthology.orchestra.run.smithi122.stdout:Complete! 2024-09-11T08:00:08.122 DEBUG:teuthology.orchestra.run.smithi122:> sudo yum -y install librados2 2024-09-11T08:00:08.678 INFO:teuthology.orchestra.run.smithi122.stdout:Last metadata expiration check: 0:01:31 ago on Wed 11 Sep 2024 07:58:37 AM UTC. 2024-09-11T08:00:08.767 INFO:teuthology.orchestra.run.smithi122.stdout:Package librados2-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T08:00:08.810 INFO:teuthology.orchestra.run.smithi122.stdout:Dependencies resolved. 2024-09-11T08:00:08.810 INFO:teuthology.orchestra.run.smithi122.stdout:Nothing to do. 2024-09-11T08:00:08.811 INFO:teuthology.orchestra.run.smithi122.stdout:Complete! 2024-09-11T08:00:08.883 DEBUG:teuthology.orchestra.run.smithi122:> sudo yum -y install librbd1 2024-09-11T08:00:09.172 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-1623.gd0e68 12/12 2024-09-11T08:00:09.172 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-1623.gd0e68 1/12 2024-09-11T08:00:09.172 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-11T08:00:09.172 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-11T08:00:09.172 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-11T08:00:09.173 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-11T08:00:09.173 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-11T08:00:09.173 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-11T08:00:09.173 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-11T08:00:09.173 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-11T08:00:09.173 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-11T08:00:09.173 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-11T08:00:09.446 INFO:teuthology.orchestra.run.smithi122.stdout:Last metadata expiration check: 0:01:32 ago on Wed 11 Sep 2024 07:58:37 AM UTC. 2024-09-11T08:00:09.536 INFO:teuthology.orchestra.run.smithi122.stdout:Package librbd1-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T08:00:09.578 INFO:teuthology.orchestra.run.smithi122.stdout:Dependencies resolved. 2024-09-11T08:00:09.579 INFO:teuthology.orchestra.run.smithi122.stdout:Nothing to do. 2024-09-11T08:00:09.579 INFO:teuthology.orchestra.run.smithi122.stdout:Complete! 2024-09-11T08:00:09.623 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-11T08:00:09.623 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-11T08:00:09.623 INFO:teuthology.orchestra.run.smithi135.stdout:Installed: 2024-09-11T08:00:09.623 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-diskprediction-local-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T08:00:09.624 INFO:teuthology.orchestra.run.smithi135.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-11T08:00:09.624 INFO:teuthology.orchestra.run.smithi135.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-11T08:00:09.624 INFO:teuthology.orchestra.run.smithi135.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-11T08:00:09.624 INFO:teuthology.orchestra.run.smithi135.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-11T08:00:09.624 INFO:teuthology.orchestra.run.smithi135.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-11T08:00:09.624 INFO:teuthology.orchestra.run.smithi135.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-11T08:00:09.624 INFO:teuthology.orchestra.run.smithi135.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-11T08:00:09.624 INFO:teuthology.orchestra.run.smithi135.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-11T08:00:09.624 INFO:teuthology.orchestra.run.smithi135.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-11T08:00:09.624 INFO:teuthology.orchestra.run.smithi135.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-11T08:00:09.624 INFO:teuthology.orchestra.run.smithi135.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-11T08:00:09.625 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-11T08:00:09.625 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2024-09-11T08:00:09.649 DEBUG:teuthology.orchestra.run.smithi122:> sudo yum -y install python3-rados 2024-09-11T08:00:10.017 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install ceph-mgr-rook 2024-09-11T08:00:10.214 INFO:teuthology.orchestra.run.smithi122.stdout:Last metadata expiration check: 0:01:33 ago on Wed 11 Sep 2024 07:58:37 AM UTC. 2024-09-11T08:00:10.305 INFO:teuthology.orchestra.run.smithi122.stdout:Package python3-rados-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T08:00:10.348 INFO:teuthology.orchestra.run.smithi122.stdout:Dependencies resolved. 2024-09-11T08:00:10.349 INFO:teuthology.orchestra.run.smithi122.stdout:Nothing to do. 2024-09-11T08:00:10.349 INFO:teuthology.orchestra.run.smithi122.stdout:Complete! 2024-09-11T08:00:10.420 DEBUG:teuthology.orchestra.run.smithi122:> sudo yum -y install python3-rgw 2024-09-11T08:00:10.592 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:01:32 ago on Wed 11 Sep 2024 07:58:38 AM UTC. 2024-09-11T08:00:10.726 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2024-09-11T08:00:10.727 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-09-11T08:00:10.727 INFO:teuthology.orchestra.run.smithi135.stdout: Package Arch Version Repository Size 2024-09-11T08:00:10.727 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-09-11T08:00:10.727 INFO:teuthology.orchestra.run.smithi135.stdout:Installing: 2024-09-11T08:00:10.727 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-rook noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 47 k 2024-09-11T08:00:10.727 INFO:teuthology.orchestra.run.smithi135.stdout:Installing dependencies: 2024-09-11T08:00:10.728 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-11T08:00:10.728 INFO:teuthology.orchestra.run.smithi135.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-11T08:00:10.728 INFO:teuthology.orchestra.run.smithi135.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-11T08:00:10.728 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-11T08:00:10.728 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-11T08:00:10.728 INFO:teuthology.orchestra.run.smithi135.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-11T08:00:10.728 INFO:teuthology.orchestra.run.smithi135.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-11T08:00:10.728 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-11T08:00:10.728 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-11T08:00:10.728 INFO:teuthology.orchestra.run.smithi135.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-11T08:00:10.728 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-11T08:00:10.728 INFO:teuthology.orchestra.run.smithi135.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-11T08:00:10.729 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-11T08:00:10.729 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction Summary 2024-09-11T08:00:10.729 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-09-11T08:00:10.729 INFO:teuthology.orchestra.run.smithi135.stdout:Install 13 Packages 2024-09-11T08:00:10.729 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-11T08:00:10.729 INFO:teuthology.orchestra.run.smithi135.stdout:Total download size: 2.2 M 2024-09-11T08:00:10.729 INFO:teuthology.orchestra.run.smithi135.stdout:Installed size: 26 M 2024-09-11T08:00:10.730 INFO:teuthology.orchestra.run.smithi135.stdout:Downloading Packages: 2024-09-11T08:00:10.983 INFO:teuthology.orchestra.run.smithi122.stdout:Last metadata expiration check: 0:01:33 ago on Wed 11 Sep 2024 07:58:37 AM UTC. 2024-09-11T08:00:11.074 INFO:teuthology.orchestra.run.smithi122.stdout:Package python3-rgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T08:00:11.099 INFO:teuthology.orchestra.run.smithi135.stdout:(1/13): python3-jsonpointer-2.0-4.el9.noarch.rp 138 kB/s | 19 kB 00:00 2024-09-11T08:00:11.117 INFO:teuthology.orchestra.run.smithi122.stdout:Dependencies resolved. 2024-09-11T08:00:11.118 INFO:teuthology.orchestra.run.smithi122.stdout:Nothing to do. 2024-09-11T08:00:11.118 INFO:teuthology.orchestra.run.smithi122.stdout:Complete! 2024-09-11T08:00:11.124 INFO:teuthology.orchestra.run.smithi135.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 156 kB/s | 26 kB 00:00 2024-09-11T08:00:11.149 INFO:teuthology.orchestra.run.smithi135.stdout:(3/13): ceph-mgr-rook-17.2.7-1623.gd0e6828a.el9 251 kB/s | 47 kB 00:00 2024-09-11T08:00:11.192 DEBUG:teuthology.orchestra.run.smithi122:> sudo yum -y install python3-cephfs 2024-09-11T08:00:11.216 INFO:teuthology.orchestra.run.smithi135.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.9 MB/s | 222 kB 00:00 2024-09-11T08:00:11.241 INFO:teuthology.orchestra.run.smithi135.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.3 MB/s | 159 kB 00:00 2024-09-11T08:00:11.275 INFO:teuthology.orchestra.run.smithi135.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 916 kB/s | 54 kB 00:00 2024-09-11T08:00:11.309 INFO:teuthology.orchestra.run.smithi135.stdout:(7/13): python3-cachetools-4.2.4-1.el9.noarch.r 481 kB/s | 32 kB 00:00 2024-09-11T08:00:11.334 INFO:teuthology.orchestra.run.smithi135.stdout:(8/13): python3-certifi-2023.05.07-4.el9.noarch 248 kB/s | 14 kB 00:00 2024-09-11T08:00:11.359 INFO:teuthology.orchestra.run.smithi135.stdout:(9/13): python3-google-auth-2.34.0-1.el9.noarch 4.4 MB/s | 220 kB 00:00 2024-09-11T08:00:11.393 INFO:teuthology.orchestra.run.smithi135.stdout:(10/13): python3-pyasn1-modules-0.4.8-6.el9.noa 1.1 MB/s | 279 kB 00:00 2024-09-11T08:00:11.418 INFO:teuthology.orchestra.run.smithi135.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-09-11T08:00:11.468 INFO:teuthology.orchestra.run.smithi135.stdout:(12/13): python3-kubernetes-26.1.0-2.el9.noarch 7.6 MB/s | 1.0 MB 00:00 2024-09-11T08:00:11.493 INFO:teuthology.orchestra.run.smithi135.stdout:(13/13): python3-websocket-client-1.2.3-2.el9.n 895 kB/s | 90 kB 00:00 2024-09-11T08:00:11.496 INFO:teuthology.orchestra.run.smithi135.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:00:11.496 INFO:teuthology.orchestra.run.smithi135.stdout:Total 2.9 MB/s | 2.2 MB 00:00 2024-09-11T08:00:11.557 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction check 2024-09-11T08:00:11.571 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction check succeeded. 2024-09-11T08:00:11.572 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction test 2024-09-11T08:00:11.660 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction test succeeded. 2024-09-11T08:00:11.661 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction 2024-09-11T08:00:11.763 INFO:teuthology.orchestra.run.smithi122.stdout:Last metadata expiration check: 0:01:34 ago on Wed 11 Sep 2024 07:58:37 AM UTC. 2024-09-11T08:00:11.854 INFO:teuthology.orchestra.run.smithi122.stdout:Package python3-cephfs-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T08:00:11.898 INFO:teuthology.orchestra.run.smithi122.stdout:Dependencies resolved. 2024-09-11T08:00:11.898 INFO:teuthology.orchestra.run.smithi122.stdout:Nothing to do. 2024-09-11T08:00:11.899 INFO:teuthology.orchestra.run.smithi122.stdout:Complete! 2024-09-11T08:00:11.970 DEBUG:teuthology.orchestra.run.smithi122:> sudo yum -y install python3-rbd 2024-09-11T08:00:12.164 INFO:teuthology.orchestra.run.smithi135.stdout: Preparing : 1/1 2024-09-11T08:00:12.341 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-11T08:00:12.394 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-11T08:00:12.459 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-11T08:00:12.495 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-11T08:00:12.536 INFO:teuthology.orchestra.run.smithi122.stdout:Last metadata expiration check: 0:01:35 ago on Wed 11 Sep 2024 07:58:37 AM UTC. 2024-09-11T08:00:12.545 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-11T08:00:12.627 INFO:teuthology.orchestra.run.smithi122.stdout:Package python3-rbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T08:00:12.649 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-11T08:00:12.670 INFO:teuthology.orchestra.run.smithi122.stdout:Dependencies resolved. 2024-09-11T08:00:12.671 INFO:teuthology.orchestra.run.smithi122.stdout:Nothing to do. 2024-09-11T08:00:12.671 INFO:teuthology.orchestra.run.smithi122.stdout:Complete! 2024-09-11T08:00:12.741 DEBUG:teuthology.orchestra.run.smithi122:> sudo yum -y install rbd-fuse 2024-09-11T08:00:12.758 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-11T08:00:12.814 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-11T08:00:13.303 INFO:teuthology.orchestra.run.smithi122.stdout:Last metadata expiration check: 0:01:36 ago on Wed 11 Sep 2024 07:58:37 AM UTC. 2024-09-11T08:00:13.437 INFO:teuthology.orchestra.run.smithi122.stdout:Dependencies resolved. 2024-09-11T08:00:13.438 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-11T08:00:13.438 INFO:teuthology.orchestra.run.smithi122.stdout: Package Architecture Version Repository Size 2024-09-11T08:00:13.438 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-11T08:00:13.438 INFO:teuthology.orchestra.run.smithi122.stdout:Installing: 2024-09-11T08:00:13.438 INFO:teuthology.orchestra.run.smithi122.stdout: rbd-fuse x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 82 k 2024-09-11T08:00:13.438 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T08:00:13.438 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction Summary 2024-09-11T08:00:13.439 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-11T08:00:13.439 INFO:teuthology.orchestra.run.smithi122.stdout:Install 1 Package 2024-09-11T08:00:13.439 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T08:00:13.439 INFO:teuthology.orchestra.run.smithi122.stdout:Total download size: 82 k 2024-09-11T08:00:13.439 INFO:teuthology.orchestra.run.smithi122.stdout:Installed size: 222 k 2024-09-11T08:00:13.439 INFO:teuthology.orchestra.run.smithi122.stdout:Downloading Packages: 2024-09-11T08:00:13.534 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-11T08:00:13.586 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-11T08:00:13.620 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-11T08:00:13.630 INFO:teuthology.orchestra.run.smithi122.stdout:rbd-fuse-17.2.7-1623.gd0e6828a.el9.x86_64.rpm 427 kB/s | 82 kB 00:00 2024-09-11T08:00:13.631 INFO:teuthology.orchestra.run.smithi122.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:00:13.631 INFO:teuthology.orchestra.run.smithi122.stdout:Total 425 kB/s | 82 kB 00:00 2024-09-11T08:00:13.631 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction check 2024-09-11T08:00:13.638 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction check succeeded. 2024-09-11T08:00:13.638 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction test 2024-09-11T08:00:13.666 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-11T08:00:13.689 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mgr-rook-2:17.2.7-1623.gd0e6828a.el9.noarch 13/13 2024-09-11T08:00:13.702 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction test succeeded. 2024-09-11T08:00:13.702 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction 2024-09-11T08:00:13.848 INFO:teuthology.orchestra.run.smithi122.stdout: Preparing : 1/1 2024-09-11T08:00:13.961 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : rbd-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:00:14.329 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: rbd-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:00:14.478 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-1623.gd0e6828a.el9.noarch 13/13 2024-09-11T08:00:14.478 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mgr-rook-2:17.2.7-1623.gd0e6828a.el9.noarch 1/13 2024-09-11T08:00:14.478 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-11T08:00:14.478 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-11T08:00:14.479 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-11T08:00:14.479 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-11T08:00:14.479 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-11T08:00:14.479 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-11T08:00:14.479 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-11T08:00:14.479 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-11T08:00:14.479 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-11T08:00:14.479 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-11T08:00:14.479 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-11T08:00:14.621 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : rbd-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:00:14.621 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T08:00:14.621 INFO:teuthology.orchestra.run.smithi122.stdout:Installed: 2024-09-11T08:00:14.621 INFO:teuthology.orchestra.run.smithi122.stdout: rbd-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:00:14.621 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T08:00:14.621 INFO:teuthology.orchestra.run.smithi122.stdout:Complete! 2024-09-11T08:00:14.804 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-11T08:00:14.804 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-11T08:00:14.804 INFO:teuthology.orchestra.run.smithi135.stdout:Installed: 2024-09-11T08:00:14.805 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-rook-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T08:00:14.805 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-11T08:00:14.805 INFO:teuthology.orchestra.run.smithi135.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-11T08:00:14.805 INFO:teuthology.orchestra.run.smithi135.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-11T08:00:14.805 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-11T08:00:14.805 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-11T08:00:14.805 INFO:teuthology.orchestra.run.smithi135.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-11T08:00:14.805 INFO:teuthology.orchestra.run.smithi135.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-11T08:00:14.805 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-11T08:00:14.805 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-11T08:00:14.805 INFO:teuthology.orchestra.run.smithi135.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-11T08:00:14.806 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-11T08:00:14.806 INFO:teuthology.orchestra.run.smithi135.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-11T08:00:14.806 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-11T08:00:14.806 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2024-09-11T08:00:14.866 DEBUG:teuthology.orchestra.run.smithi122:> sudo yum -y install rbd-mirror 2024-09-11T08:00:15.067 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install ceph-mgr-cephadm 2024-09-11T08:00:15.442 INFO:teuthology.orchestra.run.smithi122.stdout:Last metadata expiration check: 0:01:38 ago on Wed 11 Sep 2024 07:58:37 AM UTC. 2024-09-11T08:00:15.578 INFO:teuthology.orchestra.run.smithi122.stdout:Dependencies resolved. 2024-09-11T08:00:15.579 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-11T08:00:15.579 INFO:teuthology.orchestra.run.smithi122.stdout: Package Arch Version Repo Size 2024-09-11T08:00:15.579 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-11T08:00:15.580 INFO:teuthology.orchestra.run.smithi122.stdout:Installing: 2024-09-11T08:00:15.580 INFO:teuthology.orchestra.run.smithi122.stdout: rbd-mirror x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 2.9 M 2024-09-11T08:00:15.580 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T08:00:15.580 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction Summary 2024-09-11T08:00:15.580 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-11T08:00:15.580 INFO:teuthology.orchestra.run.smithi122.stdout:Install 1 Package 2024-09-11T08:00:15.580 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T08:00:15.580 INFO:teuthology.orchestra.run.smithi122.stdout:Total download size: 2.9 M 2024-09-11T08:00:15.580 INFO:teuthology.orchestra.run.smithi122.stdout:Installed size: 12 M 2024-09-11T08:00:15.581 INFO:teuthology.orchestra.run.smithi122.stdout:Downloading Packages: 2024-09-11T08:00:15.633 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:01:37 ago on Wed 11 Sep 2024 07:58:38 AM UTC. 2024-09-11T08:00:15.768 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2024-09-11T08:00:15.769 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-09-11T08:00:15.769 INFO:teuthology.orchestra.run.smithi135.stdout: Package Arch Version Repository Size 2024-09-11T08:00:15.769 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-09-11T08:00:15.769 INFO:teuthology.orchestra.run.smithi135.stdout:Installing: 2024-09-11T08:00:15.769 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-cephadm noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 124 k 2024-09-11T08:00:15.769 INFO:teuthology.orchestra.run.smithi135.stdout:Installing dependencies: 2024-09-11T08:00:15.769 INFO:teuthology.orchestra.run.smithi135.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-11T08:00:15.769 INFO:teuthology.orchestra.run.smithi135.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-11T08:00:15.769 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-11T08:00:15.769 INFO:teuthology.orchestra.run.smithi135.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-11T08:00:15.770 INFO:teuthology.orchestra.run.smithi135.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-11T08:00:15.770 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-11T08:00:15.770 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction Summary 2024-09-11T08:00:15.770 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-09-11T08:00:15.770 INFO:teuthology.orchestra.run.smithi135.stdout:Install 6 Packages 2024-09-11T08:00:15.770 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-11T08:00:15.770 INFO:teuthology.orchestra.run.smithi135.stdout:Total download size: 7.0 M 2024-09-11T08:00:15.771 INFO:teuthology.orchestra.run.smithi135.stdout:Installed size: 33 M 2024-09-11T08:00:15.771 INFO:teuthology.orchestra.run.smithi135.stdout:Downloading Packages: 2024-09-11T08:00:15.926 INFO:teuthology.orchestra.run.smithi122.stdout:rbd-mirror-17.2.7-1623.gd0e6828a.el9.x86_64.rpm 8.5 MB/s | 2.9 MB 00:00 2024-09-11T08:00:15.927 INFO:teuthology.orchestra.run.smithi122.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:00:15.928 INFO:teuthology.orchestra.run.smithi122.stdout:Total 8.5 MB/s | 2.9 MB 00:00 2024-09-11T08:00:15.928 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction check 2024-09-11T08:00:15.936 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction check succeeded. 2024-09-11T08:00:15.937 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction test 2024-09-11T08:00:16.015 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction test succeeded. 2024-09-11T08:00:16.015 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction 2024-09-11T08:00:16.253 INFO:teuthology.orchestra.run.smithi135.stdout:(1/6): ceph-mgr-cephadm-17.2.7-1623.gd0e6828a.e 614 kB/s | 124 kB 00:00 2024-09-11T08:00:16.345 INFO:teuthology.orchestra.run.smithi135.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 5.8 MB/s | 548 kB 00:00 2024-09-11T08:00:16.370 INFO:teuthology.orchestra.run.smithi135.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 2.2 MB/s | 58 kB 00:00 2024-09-11T08:00:16.404 INFO:teuthology.orchestra.run.smithi135.stdout:(4/6): python3-typing-extensions-4.12.2-2.el9.n 2.2 MB/s | 74 kB 00:00 2024-09-11T08:00:16.431 INFO:teuthology.orchestra.run.smithi122.stdout: Preparing : 1/1 2024-09-11T08:00:16.511 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : rbd-mirror-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:00:16.535 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: rbd-mirror-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:00:16.536 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T08:00:16.536 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-11T08:00:16.536 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T08:00:16.536 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T08:00:16.536 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T08:00:16.596 INFO:teuthology.orchestra.run.smithi135.stdout:(5/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 456 kB/s | 248 kB 00:00 2024-09-11T08:00:17.296 INFO:teuthology.orchestra.run.smithi135.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 4.8 MB/s | 6.0 MB 00:01 2024-09-11T08:00:17.298 INFO:teuthology.orchestra.run.smithi135.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:00:17.299 INFO:teuthology.orchestra.run.smithi135.stdout:Total 4.6 MB/s | 7.0 MB 00:01 2024-09-11T08:00:17.371 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction check 2024-09-11T08:00:17.382 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction check succeeded. 2024-09-11T08:00:17.383 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction test 2024-09-11T08:00:17.484 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction test succeeded. 2024-09-11T08:00:17.484 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction 2024-09-11T08:00:17.644 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : rbd-mirror-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:00:17.644 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T08:00:17.644 INFO:teuthology.orchestra.run.smithi122.stdout:Installed: 2024-09-11T08:00:17.644 INFO:teuthology.orchestra.run.smithi122.stdout: rbd-mirror-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:00:17.644 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T08:00:17.644 INFO:teuthology.orchestra.run.smithi122.stdout:Complete! 2024-09-11T08:00:17.785 INFO:teuthology.orchestra.run.smithi135.stdout: Preparing : 1/1 2024-09-11T08:00:17.863 DEBUG:teuthology.orchestra.run.smithi122:> sudo yum -y install rbd-nbd 2024-09-11T08:00:17.962 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-11T08:00:18.006 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-11T08:00:18.431 INFO:teuthology.orchestra.run.smithi122.stdout:Last metadata expiration check: 0:01:41 ago on Wed 11 Sep 2024 07:58:37 AM UTC. 2024-09-11T08:00:18.530 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-11T08:00:18.567 INFO:teuthology.orchestra.run.smithi122.stdout:Dependencies resolved. 2024-09-11T08:00:18.567 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-11T08:00:18.567 INFO:teuthology.orchestra.run.smithi122.stdout: Package Architecture Version Repository Size 2024-09-11T08:00:18.567 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-11T08:00:18.567 INFO:teuthology.orchestra.run.smithi122.stdout:Installing: 2024-09-11T08:00:18.567 INFO:teuthology.orchestra.run.smithi122.stdout: rbd-nbd x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 165 k 2024-09-11T08:00:18.567 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T08:00:18.567 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction Summary 2024-09-11T08:00:18.567 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-11T08:00:18.567 INFO:teuthology.orchestra.run.smithi122.stdout:Install 1 Package 2024-09-11T08:00:18.568 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T08:00:18.568 INFO:teuthology.orchestra.run.smithi122.stdout:Total download size: 165 k 2024-09-11T08:00:18.568 INFO:teuthology.orchestra.run.smithi122.stdout:Installed size: 477 k 2024-09-11T08:00:18.568 INFO:teuthology.orchestra.run.smithi122.stdout:Downloading Packages: 2024-09-11T08:00:18.626 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-11T08:00:18.681 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-11T08:00:18.710 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 6/6 2024-09-11T08:00:18.780 INFO:teuthology.orchestra.run.smithi122.stdout:rbd-nbd-17.2.7-1623.gd0e6828a.el9.x86_64.rpm 778 kB/s | 165 kB 00:00 2024-09-11T08:00:18.781 INFO:teuthology.orchestra.run.smithi122.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:00:18.781 INFO:teuthology.orchestra.run.smithi122.stdout:Total 774 kB/s | 165 kB 00:00 2024-09-11T08:00:18.781 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction check 2024-09-11T08:00:18.788 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction check succeeded. 2024-09-11T08:00:18.788 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction test 2024-09-11T08:00:18.852 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction test succeeded. 2024-09-11T08:00:18.852 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction 2024-09-11T08:00:19.007 INFO:teuthology.orchestra.run.smithi122.stdout: Preparing : 1/1 2024-09-11T08:00:19.119 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : rbd-nbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:00:19.508 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: rbd-nbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:00:19.546 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 6/6 2024-09-11T08:00:19.546 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 1/6 2024-09-11T08:00:19.546 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-11T08:00:19.546 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-11T08:00:19.546 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-11T08:00:19.546 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-11T08:00:19.729 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : rbd-nbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:00:19.730 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T08:00:19.730 INFO:teuthology.orchestra.run.smithi122.stdout:Installed: 2024-09-11T08:00:19.730 INFO:teuthology.orchestra.run.smithi122.stdout: rbd-nbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:00:19.730 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T08:00:19.730 INFO:teuthology.orchestra.run.smithi122.stdout:Complete! 2024-09-11T08:00:19.819 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-11T08:00:19.819 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-11T08:00:19.819 INFO:teuthology.orchestra.run.smithi135.stdout:Installed: 2024-09-11T08:00:19.819 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T08:00:19.819 INFO:teuthology.orchestra.run.smithi135.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-11T08:00:19.819 INFO:teuthology.orchestra.run.smithi135.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-11T08:00:19.819 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-11T08:00:19.819 INFO:teuthology.orchestra.run.smithi135.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-11T08:00:19.819 INFO:teuthology.orchestra.run.smithi135.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-11T08:00:19.819 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-11T08:00:19.819 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2024-09-11T08:00:19.976 DEBUG:teuthology.parallel:result is None 2024-09-11T08:00:20.077 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install ceph-fuse 2024-09-11T08:00:20.647 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:01:42 ago on Wed 11 Sep 2024 07:58:38 AM UTC. 2024-09-11T08:00:20.782 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2024-09-11T08:00:20.784 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-09-11T08:00:20.784 INFO:teuthology.orchestra.run.smithi135.stdout: Package Arch Version Repository Size 2024-09-11T08:00:20.784 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-09-11T08:00:20.784 INFO:teuthology.orchestra.run.smithi135.stdout:Installing: 2024-09-11T08:00:20.784 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-fuse x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 813 k 2024-09-11T08:00:20.784 INFO:teuthology.orchestra.run.smithi135.stdout:Installing dependencies: 2024-09-11T08:00:20.784 INFO:teuthology.orchestra.run.smithi135.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-11T08:00:20.785 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-11T08:00:20.785 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction Summary 2024-09-11T08:00:20.786 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-09-11T08:00:20.786 INFO:teuthology.orchestra.run.smithi135.stdout:Install 2 Packages 2024-09-11T08:00:20.786 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-11T08:00:20.786 INFO:teuthology.orchestra.run.smithi135.stdout:Total download size: 893 k 2024-09-11T08:00:20.786 INFO:teuthology.orchestra.run.smithi135.stdout:Installed size: 2.6 M 2024-09-11T08:00:20.786 INFO:teuthology.orchestra.run.smithi135.stdout:Downloading Packages: 2024-09-11T08:00:20.914 INFO:teuthology.orchestra.run.smithi135.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 748 kB/s | 80 kB 00:00 2024-09-11T08:00:21.098 INFO:teuthology.orchestra.run.smithi135.stdout:(2/2): ceph-fuse-17.2.7-1623.gd0e6828a.el9.x86_ 2.7 MB/s | 813 kB 00:00 2024-09-11T08:00:21.098 INFO:teuthology.orchestra.run.smithi135.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:00:21.099 INFO:teuthology.orchestra.run.smithi135.stdout:Total 2.8 MB/s | 893 kB 00:00 2024-09-11T08:00:21.110 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction check 2024-09-11T08:00:21.132 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction check succeeded. 2024-09-11T08:00:21.132 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction test 2024-09-11T08:00:21.217 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction test succeeded. 2024-09-11T08:00:21.217 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction 2024-09-11T08:00:21.408 INFO:teuthology.orchestra.run.smithi135.stdout: Preparing : 1/1 2024-09-11T08:00:21.555 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-11T08:00:21.630 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/2 2024-09-11T08:00:22.318 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/2 2024-09-11T08:00:22.318 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/2 2024-09-11T08:00:22.620 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-11T08:00:22.620 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-11T08:00:22.620 INFO:teuthology.orchestra.run.smithi135.stdout:Installed: 2024-09-11T08:00:22.620 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-11T08:00:22.620 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-11T08:00:22.620 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2024-09-11T08:00:22.903 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install ceph-volume 2024-09-11T08:00:23.485 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:01:45 ago on Wed 11 Sep 2024 07:58:38 AM UTC. 2024-09-11T08:00:23.622 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2024-09-11T08:00:23.622 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-09-11T08:00:23.623 INFO:teuthology.orchestra.run.smithi135.stdout: Package Arch Version Repository Size 2024-09-11T08:00:23.623 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-09-11T08:00:23.623 INFO:teuthology.orchestra.run.smithi135.stdout:Installing: 2024-09-11T08:00:23.623 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-volume noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 270 k 2024-09-11T08:00:23.623 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-11T08:00:23.623 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction Summary 2024-09-11T08:00:23.623 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-09-11T08:00:23.623 INFO:teuthology.orchestra.run.smithi135.stdout:Install 1 Package 2024-09-11T08:00:23.623 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-11T08:00:23.623 INFO:teuthology.orchestra.run.smithi135.stdout:Total download size: 270 k 2024-09-11T08:00:23.623 INFO:teuthology.orchestra.run.smithi135.stdout:Installed size: 1.3 M 2024-09-11T08:00:23.623 INFO:teuthology.orchestra.run.smithi135.stdout:Downloading Packages: 2024-09-11T08:00:23.852 INFO:teuthology.orchestra.run.smithi135.stdout:ceph-volume-17.2.7-1623.gd0e6828a.el9.noarch.rp 1.2 MB/s | 270 kB 00:00 2024-09-11T08:00:23.853 INFO:teuthology.orchestra.run.smithi135.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:00:23.853 INFO:teuthology.orchestra.run.smithi135.stdout:Total 1.1 MB/s | 270 kB 00:00 2024-09-11T08:00:23.853 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction check 2024-09-11T08:00:23.862 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction check succeeded. 2024-09-11T08:00:23.862 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction test 2024-09-11T08:00:23.891 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction test succeeded. 2024-09-11T08:00:23.891 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction 2024-09-11T08:00:24.104 INFO:teuthology.orchestra.run.smithi135.stdout: Preparing : 1/1 2024-09-11T08:00:24.171 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-volume-2:17.2.7-1623.gd0e6828a.el9.noarch 1/1 2024-09-11T08:00:24.187 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-volume-2:17.2.7-1623.gd0e6828a.el9.noarch 1/1 2024-09-11T08:00:24.188 INFO:teuthology.orchestra.run.smithi135.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T08:00:24.188 INFO:teuthology.orchestra.run.smithi135.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-11T08:00:24.188 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-11T08:00:25.239 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-volume-2:17.2.7-1623.gd0e6828a.el9.noarch 1/1 2024-09-11T08:00:25.239 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-11T08:00:25.239 INFO:teuthology.orchestra.run.smithi135.stdout:Installed: 2024-09-11T08:00:25.239 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-volume-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T08:00:25.239 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-11T08:00:25.239 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2024-09-11T08:00:25.546 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install librados-devel 2024-09-11T08:00:26.122 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:01:48 ago on Wed 11 Sep 2024 07:58:38 AM UTC. 2024-09-11T08:00:26.257 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2024-09-11T08:00:26.257 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-09-11T08:00:26.257 INFO:teuthology.orchestra.run.smithi135.stdout: Package Arch Version Repo Size 2024-09-11T08:00:26.258 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-09-11T08:00:26.258 INFO:teuthology.orchestra.run.smithi135.stdout:Installing: 2024-09-11T08:00:26.258 INFO:teuthology.orchestra.run.smithi135.stdout: librados-devel x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 126 k 2024-09-11T08:00:26.258 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-11T08:00:26.258 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction Summary 2024-09-11T08:00:26.258 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-09-11T08:00:26.258 INFO:teuthology.orchestra.run.smithi135.stdout:Install 1 Package 2024-09-11T08:00:26.258 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-11T08:00:26.258 INFO:teuthology.orchestra.run.smithi135.stdout:Total download size: 126 k 2024-09-11T08:00:26.258 INFO:teuthology.orchestra.run.smithi135.stdout:Installed size: 448 k 2024-09-11T08:00:26.258 INFO:teuthology.orchestra.run.smithi135.stdout:Downloading Packages: 2024-09-11T08:00:26.474 INFO:teuthology.orchestra.run.smithi135.stdout:librados-devel-17.2.7-1623.gd0e6828a.el9.x86_64 585 kB/s | 126 kB 00:00 2024-09-11T08:00:26.475 INFO:teuthology.orchestra.run.smithi135.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:00:26.475 INFO:teuthology.orchestra.run.smithi135.stdout:Total 581 kB/s | 126 kB 00:00 2024-09-11T08:00:26.475 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction check 2024-09-11T08:00:26.481 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction check succeeded. 2024-09-11T08:00:26.481 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction test 2024-09-11T08:00:26.544 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction test succeeded. 2024-09-11T08:00:26.545 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction 2024-09-11T08:00:26.691 INFO:teuthology.orchestra.run.smithi135.stdout: Preparing : 1/1 2024-09-11T08:00:26.779 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : librados-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:00:27.209 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: librados-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:00:27.469 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librados-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:00:27.470 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-11T08:00:27.470 INFO:teuthology.orchestra.run.smithi135.stdout:Installed: 2024-09-11T08:00:27.470 INFO:teuthology.orchestra.run.smithi135.stdout: librados-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:00:27.470 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-11T08:00:27.470 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2024-09-11T08:00:27.734 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install libcephfs2 2024-09-11T08:00:28.317 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:01:50 ago on Wed 11 Sep 2024 07:58:38 AM UTC. 2024-09-11T08:00:28.409 INFO:teuthology.orchestra.run.smithi135.stdout:Package libcephfs2-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T08:00:28.454 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2024-09-11T08:00:28.455 INFO:teuthology.orchestra.run.smithi135.stdout:Nothing to do. 2024-09-11T08:00:28.455 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2024-09-11T08:00:28.546 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install libcephfs-devel 2024-09-11T08:00:29.117 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:01:51 ago on Wed 11 Sep 2024 07:58:38 AM UTC. 2024-09-11T08:00:29.251 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2024-09-11T08:00:29.252 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-09-11T08:00:29.252 INFO:teuthology.orchestra.run.smithi135.stdout: Package Arch Version Repo Size 2024-09-11T08:00:29.252 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-09-11T08:00:29.252 INFO:teuthology.orchestra.run.smithi135.stdout:Installing: 2024-09-11T08:00:29.252 INFO:teuthology.orchestra.run.smithi135.stdout: libcephfs-devel x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 25 k 2024-09-11T08:00:29.252 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-11T08:00:29.252 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction Summary 2024-09-11T08:00:29.253 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-09-11T08:00:29.253 INFO:teuthology.orchestra.run.smithi135.stdout:Install 1 Package 2024-09-11T08:00:29.253 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-11T08:00:29.253 INFO:teuthology.orchestra.run.smithi135.stdout:Total download size: 25 k 2024-09-11T08:00:29.253 INFO:teuthology.orchestra.run.smithi135.stdout:Installed size: 110 k 2024-09-11T08:00:29.253 INFO:teuthology.orchestra.run.smithi135.stdout:Downloading Packages: 2024-09-11T08:00:29.411 INFO:teuthology.orchestra.run.smithi135.stdout:libcephfs-devel-17.2.7-1623.gd0e6828a.el9.x86_6 157 kB/s | 25 kB 00:00 2024-09-11T08:00:29.412 INFO:teuthology.orchestra.run.smithi135.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:00:29.412 INFO:teuthology.orchestra.run.smithi135.stdout:Total 156 kB/s | 25 kB 00:00 2024-09-11T08:00:29.412 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction check 2024-09-11T08:00:29.416 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction check succeeded. 2024-09-11T08:00:29.416 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction test 2024-09-11T08:00:29.417 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction test succeeded. 2024-09-11T08:00:29.418 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction 2024-09-11T08:00:29.504 INFO:teuthology.orchestra.run.smithi135.stdout: Preparing : 1/1 2024-09-11T08:00:29.608 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libcephfs-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:00:29.983 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:00:30.225 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libcephfs-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:00:30.226 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-11T08:00:30.226 INFO:teuthology.orchestra.run.smithi135.stdout:Installed: 2024-09-11T08:00:30.226 INFO:teuthology.orchestra.run.smithi135.stdout: libcephfs-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:00:30.226 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-11T08:00:30.226 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2024-09-11T08:00:30.404 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install librados2 2024-09-11T08:00:30.992 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:01:52 ago on Wed 11 Sep 2024 07:58:38 AM UTC. 2024-09-11T08:00:31.086 INFO:teuthology.orchestra.run.smithi135.stdout:Package librados2-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T08:00:31.131 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2024-09-11T08:00:31.132 INFO:teuthology.orchestra.run.smithi135.stdout:Nothing to do. 2024-09-11T08:00:31.133 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2024-09-11T08:00:31.206 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install librbd1 2024-09-11T08:00:31.804 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:01:53 ago on Wed 11 Sep 2024 07:58:38 AM UTC. 2024-09-11T08:00:31.897 INFO:teuthology.orchestra.run.smithi135.stdout:Package librbd1-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T08:00:31.941 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2024-09-11T08:00:31.942 INFO:teuthology.orchestra.run.smithi135.stdout:Nothing to do. 2024-09-11T08:00:31.943 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2024-09-11T08:00:32.016 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install python3-rados 2024-09-11T08:00:32.591 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:01:54 ago on Wed 11 Sep 2024 07:58:38 AM UTC. 2024-09-11T08:00:32.683 INFO:teuthology.orchestra.run.smithi135.stdout:Package python3-rados-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T08:00:32.725 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2024-09-11T08:00:32.726 INFO:teuthology.orchestra.run.smithi135.stdout:Nothing to do. 2024-09-11T08:00:32.727 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2024-09-11T08:00:32.794 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install python3-rgw 2024-09-11T08:00:33.362 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:01:55 ago on Wed 11 Sep 2024 07:58:38 AM UTC. 2024-09-11T08:00:33.453 INFO:teuthology.orchestra.run.smithi135.stdout:Package python3-rgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T08:00:33.495 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2024-09-11T08:00:33.496 INFO:teuthology.orchestra.run.smithi135.stdout:Nothing to do. 2024-09-11T08:00:33.496 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2024-09-11T08:00:33.560 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install python3-cephfs 2024-09-11T08:00:34.134 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:01:56 ago on Wed 11 Sep 2024 07:58:38 AM UTC. 2024-09-11T08:00:34.224 INFO:teuthology.orchestra.run.smithi135.stdout:Package python3-cephfs-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T08:00:34.266 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2024-09-11T08:00:34.267 INFO:teuthology.orchestra.run.smithi135.stdout:Nothing to do. 2024-09-11T08:00:34.267 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2024-09-11T08:00:34.341 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install python3-rbd 2024-09-11T08:00:34.912 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:01:56 ago on Wed 11 Sep 2024 07:58:38 AM UTC. 2024-09-11T08:00:35.002 INFO:teuthology.orchestra.run.smithi135.stdout:Package python3-rbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T08:00:35.045 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2024-09-11T08:00:35.046 INFO:teuthology.orchestra.run.smithi135.stdout:Nothing to do. 2024-09-11T08:00:35.046 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2024-09-11T08:00:35.108 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install rbd-fuse 2024-09-11T08:00:35.673 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:01:57 ago on Wed 11 Sep 2024 07:58:38 AM UTC. 2024-09-11T08:00:35.808 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2024-09-11T08:00:35.809 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-09-11T08:00:35.809 INFO:teuthology.orchestra.run.smithi135.stdout: Package Architecture Version Repository Size 2024-09-11T08:00:35.809 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-09-11T08:00:35.809 INFO:teuthology.orchestra.run.smithi135.stdout:Installing: 2024-09-11T08:00:35.809 INFO:teuthology.orchestra.run.smithi135.stdout: rbd-fuse x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 82 k 2024-09-11T08:00:35.809 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-11T08:00:35.809 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction Summary 2024-09-11T08:00:35.810 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-09-11T08:00:35.810 INFO:teuthology.orchestra.run.smithi135.stdout:Install 1 Package 2024-09-11T08:00:35.810 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-11T08:00:35.810 INFO:teuthology.orchestra.run.smithi135.stdout:Total download size: 82 k 2024-09-11T08:00:35.810 INFO:teuthology.orchestra.run.smithi135.stdout:Installed size: 222 k 2024-09-11T08:00:35.810 INFO:teuthology.orchestra.run.smithi135.stdout:Downloading Packages: 2024-09-11T08:00:36.007 INFO:teuthology.orchestra.run.smithi135.stdout:rbd-fuse-17.2.7-1623.gd0e6828a.el9.x86_64.rpm 415 kB/s | 82 kB 00:00 2024-09-11T08:00:36.008 INFO:teuthology.orchestra.run.smithi135.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:00:36.008 INFO:teuthology.orchestra.run.smithi135.stdout:Total 413 kB/s | 82 kB 00:00 2024-09-11T08:00:36.008 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction check 2024-09-11T08:00:36.014 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction check succeeded. 2024-09-11T08:00:36.014 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction test 2024-09-11T08:00:36.077 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction test succeeded. 2024-09-11T08:00:36.077 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction 2024-09-11T08:00:36.230 INFO:teuthology.orchestra.run.smithi135.stdout: Preparing : 1/1 2024-09-11T08:00:36.329 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : rbd-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:00:36.731 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: rbd-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:00:36.977 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : rbd-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:00:36.978 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-11T08:00:36.978 INFO:teuthology.orchestra.run.smithi135.stdout:Installed: 2024-09-11T08:00:36.978 INFO:teuthology.orchestra.run.smithi135.stdout: rbd-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:00:36.978 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-11T08:00:36.978 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2024-09-11T08:00:37.249 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install rbd-mirror 2024-09-11T08:00:37.832 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:01:59 ago on Wed 11 Sep 2024 07:58:38 AM UTC. 2024-09-11T08:00:37.971 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2024-09-11T08:00:37.972 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-09-11T08:00:37.972 INFO:teuthology.orchestra.run.smithi135.stdout: Package Arch Version Repo Size 2024-09-11T08:00:37.972 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-09-11T08:00:37.972 INFO:teuthology.orchestra.run.smithi135.stdout:Installing: 2024-09-11T08:00:37.972 INFO:teuthology.orchestra.run.smithi135.stdout: rbd-mirror x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 2.9 M 2024-09-11T08:00:37.972 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-11T08:00:37.972 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction Summary 2024-09-11T08:00:37.972 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-09-11T08:00:37.972 INFO:teuthology.orchestra.run.smithi135.stdout:Install 1 Package 2024-09-11T08:00:37.972 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-11T08:00:37.972 INFO:teuthology.orchestra.run.smithi135.stdout:Total download size: 2.9 M 2024-09-11T08:00:37.972 INFO:teuthology.orchestra.run.smithi135.stdout:Installed size: 12 M 2024-09-11T08:00:37.973 INFO:teuthology.orchestra.run.smithi135.stdout:Downloading Packages: 2024-09-11T08:00:38.310 INFO:teuthology.orchestra.run.smithi135.stdout:rbd-mirror-17.2.7-1623.gd0e6828a.el9.x86_64.rpm 8.7 MB/s | 2.9 MB 00:00 2024-09-11T08:00:38.311 INFO:teuthology.orchestra.run.smithi135.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:00:38.311 INFO:teuthology.orchestra.run.smithi135.stdout:Total 8.7 MB/s | 2.9 MB 00:00 2024-09-11T08:00:38.312 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction check 2024-09-11T08:00:38.320 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction check succeeded. 2024-09-11T08:00:38.320 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction test 2024-09-11T08:00:38.402 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction test succeeded. 2024-09-11T08:00:38.402 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction 2024-09-11T08:00:38.825 INFO:teuthology.orchestra.run.smithi135.stdout: Preparing : 1/1 2024-09-11T08:00:38.994 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : rbd-mirror-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:00:39.021 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: rbd-mirror-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:00:39.021 INFO:teuthology.orchestra.run.smithi135.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T08:00:39.021 INFO:teuthology.orchestra.run.smithi135.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-11T08:00:39.021 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T08:00:39.021 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T08:00:39.021 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-11T08:00:40.042 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : rbd-mirror-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:00:40.042 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-11T08:00:40.042 INFO:teuthology.orchestra.run.smithi135.stdout:Installed: 2024-09-11T08:00:40.042 INFO:teuthology.orchestra.run.smithi135.stdout: rbd-mirror-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:00:40.042 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-11T08:00:40.042 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2024-09-11T08:00:40.297 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install rbd-nbd 2024-09-11T08:00:40.881 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:02:02 ago on Wed 11 Sep 2024 07:58:38 AM UTC. 2024-09-11T08:00:41.013 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2024-09-11T08:00:41.014 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-09-11T08:00:41.014 INFO:teuthology.orchestra.run.smithi135.stdout: Package Architecture Version Repository Size 2024-09-11T08:00:41.014 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-09-11T08:00:41.015 INFO:teuthology.orchestra.run.smithi135.stdout:Installing: 2024-09-11T08:00:41.015 INFO:teuthology.orchestra.run.smithi135.stdout: rbd-nbd x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 165 k 2024-09-11T08:00:41.015 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-11T08:00:41.015 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction Summary 2024-09-11T08:00:41.015 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-09-11T08:00:41.015 INFO:teuthology.orchestra.run.smithi135.stdout:Install 1 Package 2024-09-11T08:00:41.015 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-11T08:00:41.015 INFO:teuthology.orchestra.run.smithi135.stdout:Total download size: 165 k 2024-09-11T08:00:41.015 INFO:teuthology.orchestra.run.smithi135.stdout:Installed size: 477 k 2024-09-11T08:00:41.015 INFO:teuthology.orchestra.run.smithi135.stdout:Downloading Packages: 2024-09-11T08:00:41.222 INFO:teuthology.orchestra.run.smithi135.stdout:rbd-nbd-17.2.7-1623.gd0e6828a.el9.x86_64.rpm 798 kB/s | 165 kB 00:00 2024-09-11T08:00:41.223 INFO:teuthology.orchestra.run.smithi135.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:00:41.223 INFO:teuthology.orchestra.run.smithi135.stdout:Total 792 kB/s | 165 kB 00:00 2024-09-11T08:00:41.223 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction check 2024-09-11T08:00:41.232 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction check succeeded. 2024-09-11T08:00:41.233 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction test 2024-09-11T08:00:41.295 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction test succeeded. 2024-09-11T08:00:41.295 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction 2024-09-11T08:00:41.444 INFO:teuthology.orchestra.run.smithi135.stdout: Preparing : 1/1 2024-09-11T08:00:41.553 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : rbd-nbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:00:41.999 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: rbd-nbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:00:42.274 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : rbd-nbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:00:42.275 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-11T08:00:42.275 INFO:teuthology.orchestra.run.smithi135.stdout:Installed: 2024-09-11T08:00:42.275 INFO:teuthology.orchestra.run.smithi135.stdout: rbd-nbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:00:42.275 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-11T08:00:42.275 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2024-09-11T08:00:42.511 DEBUG:teuthology.parallel:result is None 2024-09-11T08:00:42.512 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d0e6828a2016d48cf25ad84064e50742bb1c39b9 2024-09-11T08:00:42.665 DEBUG:teuthology.orchestra.run.smithi122:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-11T08:00:42.691 INFO:teuthology.orchestra.run.smithi122.stdout:17.2.7-1623.gd0e6828a.el9 2024-09-11T08:00:42.692 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-1623.gd0e6828a.el9 2024-09-11T08:00:42.692 INFO:teuthology.task.install:The correct ceph version 17.2.7-1623.gd0e6828a is installed. 2024-09-11T08:00:42.694 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d0e6828a2016d48cf25ad84064e50742bb1c39b9 2024-09-11T08:00:42.838 DEBUG:teuthology.orchestra.run.smithi135:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-11T08:00:42.860 INFO:teuthology.orchestra.run.smithi135.stdout:17.2.7-1623.gd0e6828a.el9 2024-09-11T08:00:42.860 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-1623.gd0e6828a.el9 2024-09-11T08:00:42.860 INFO:teuthology.task.install:The correct ceph version 17.2.7-1623.gd0e6828a is installed. 2024-09-11T08:00:42.862 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-11T08:00:42.862 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-11T08:00:42.862 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-11T08:00:42.893 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-09-11T08:00:42.893 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-11T08:00:42.926 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-11T08:00:42.926 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-11T08:00:42.926 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/usr/bin/daemon-helper 2024-09-11T08:00:42.963 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-11T08:00:43.033 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-09-11T08:00:43.033 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/usr/bin/daemon-helper 2024-09-11T08:00:43.059 DEBUG:teuthology.orchestra.run.smithi135:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-11T08:00:43.124 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-11T08:00:43.124 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-11T08:00:43.124 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-11T08:00:43.153 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-11T08:00:43.221 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-09-11T08:00:43.222 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-11T08:00:43.254 DEBUG:teuthology.orchestra.run.smithi135:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-11T08:00:43.319 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-11T08:00:43.319 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-11T08:00:43.319 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/usr/bin/stdin-killer 2024-09-11T08:00:43.347 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-11T08:00:43.414 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-09-11T08:00:43.414 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/usr/bin/stdin-killer 2024-09-11T08:00:43.439 DEBUG:teuthology.orchestra.run.smithi135:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-11T08:00:43.503 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-11T08:00:43.587 INFO:tasks.cephadm:Config: {'conf': {'mgr': {'debug mgr': 20, 'debug ms': 1}, 'global': {'mon election default strategy': 1, 'ms type': 'async'}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'debug ms': 1, 'debug osd': 20, 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': 'd0e6828a2016d48cf25ad84064e50742bb1c39b9', 'cephadm_mode': 'cephadm-package'} 2024-09-11T08:00:43.588 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 2024-09-11T08:00:43.588 INFO:tasks.cephadm:Cluster fsid is f1d75850-7013-11ef-bcea-c7b262605968 2024-09-11T08:00:43.588 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-11T08:00:43.589 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.122', 'mon.c': '[v2:172.21.15.122:3301,v1:172.21.15.122:6790]', 'mon.b': '172.21.15.135'} 2024-09-11T08:00:43.589 INFO:tasks.cephadm:First mon is mon.a on smithi122 2024-09-11T08:00:43.589 INFO:tasks.cephadm:First mgr is y 2024-09-11T08:00:43.589 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-11T08:00:43.589 DEBUG:teuthology.orchestra.run.smithi122:> sudo hostname $(hostname -s) 2024-09-11T08:00:43.618 DEBUG:teuthology.orchestra.run.smithi135:> sudo hostname $(hostname -s) 2024-09-11T08:00:43.653 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 on all hosts... 2024-09-11T08:00:43.653 DEBUG:teuthology.orchestra.run.smithi122:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 pull 2024-09-11T08:00:43.664 DEBUG:teuthology.orchestra.run.smithi135:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 pull 2024-09-11T08:00:45.192 INFO:teuthology.orchestra.run.smithi122.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9... 2024-09-11T08:00:45.446 INFO:teuthology.orchestra.run.smithi135.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9... 2024-09-11T08:01:12.065 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-09-11T08:01:12.066 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_version": "ceph version 17.2.7-1623-gd0e6828a (d0e6828a2016d48cf25ad84064e50742bb1c39b9) quincy (stable)", 2024-09-11T08:01:12.066 INFO:teuthology.orchestra.run.smithi135.stdout: "image_id": "4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b", 2024-09-11T08:01:12.066 INFO:teuthology.orchestra.run.smithi135.stdout: "repo_digests": [ 2024-09-11T08:01:12.066 INFO:teuthology.orchestra.run.smithi135.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40" 2024-09-11T08:01:12.066 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-09-11T08:01:12.066 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-09-11T08:01:23.863 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-11T08:01:23.863 INFO:teuthology.orchestra.run.smithi122.stdout: "ceph_version": "ceph version 17.2.7-1623-gd0e6828a (d0e6828a2016d48cf25ad84064e50742bb1c39b9) quincy (stable)", 2024-09-11T08:01:23.863 INFO:teuthology.orchestra.run.smithi122.stdout: "image_id": "4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b", 2024-09-11T08:01:23.863 INFO:teuthology.orchestra.run.smithi122.stdout: "repo_digests": [ 2024-09-11T08:01:23.863 INFO:teuthology.orchestra.run.smithi122.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40" 2024-09-11T08:01:23.863 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-11T08:01:23.863 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-11T08:01:23.883 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /etc/ceph 2024-09-11T08:01:23.926 DEBUG:teuthology.orchestra.run.smithi135:> sudo mkdir -p /etc/ceph 2024-09-11T08:01:23.959 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 777 /etc/ceph 2024-09-11T08:01:23.992 DEBUG:teuthology.orchestra.run.smithi135:> sudo chmod 777 /etc/ceph 2024-09-11T08:01:24.027 INFO:tasks.cephadm:Writing seed config... 2024-09-11T08:01:24.029 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-11T08:01:24.029 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-11T08:01:24.029 INFO:tasks.cephadm: override: [global] mon election default strategy = 1 2024-09-11T08:01:24.029 INFO:tasks.cephadm: override: [global] ms type = async 2024-09-11T08:01:24.029 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-11T08:01:24.029 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-11T08:01:24.029 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-11T08:01:24.029 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-11T08:01:24.029 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-11T08:01:24.032 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-11T08:01:24.032 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-11T08:01:24.032 DEBUG:teuthology.orchestra.run.smithi122:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-11T08:01:24.048 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = f1d75850-7013-11ef-bcea-c7b262605968 mon election default strategy = 1 ms type = async [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true debug ms = 1 debug osd = 20 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true 2024-09-11T08:01:24.049 DEBUG:teuthology.orchestra.run.smithi122:mon.a> sudo journalctl -f -n 0 -u ceph-f1d75850-7013-11ef-bcea-c7b262605968@mon.a.service 2024-09-11T08:01:24.091 DEBUG:teuthology.orchestra.run.smithi122:mgr.y> sudo journalctl -f -n 0 -u ceph-f1d75850-7013-11ef-bcea-c7b262605968@mgr.y.service 2024-09-11T08:01:24.133 INFO:tasks.cephadm:Bootstrapping... 2024-09-11T08:01:24.133 DEBUG:teuthology.orchestra.run.smithi122:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 -v bootstrap --fsid f1d75850-7013-11ef-bcea-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id y --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.122 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-11T08:01:24.330 INFO:teuthology.orchestra.run.smithi122.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:01:24.330 INFO:teuthology.orchestra.run.smithi122.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9', '-v', 'bootstrap', '--fsid', 'f1d75850-7013-11ef-bcea-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'y', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.122', '--skip-admin-label'] 2024-09-11T08:01:24.354 INFO:teuthology.orchestra.run.smithi122.stdout:/bin/podman: stdout 5.2.2 2024-09-11T08:01:24.354 INFO:teuthology.orchestra.run.smithi122.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-11T08:01:24.355 INFO:teuthology.orchestra.run.smithi122.stdout:Verifying podman|docker is present... 2024-09-11T08:01:24.377 INFO:teuthology.orchestra.run.smithi122.stdout:/bin/podman: stdout 5.2.2 2024-09-11T08:01:24.377 INFO:teuthology.orchestra.run.smithi122.stdout:Verifying lvm2 is present... 2024-09-11T08:01:24.377 INFO:teuthology.orchestra.run.smithi122.stdout:Verifying time synchronization is in place... 2024-09-11T08:01:24.385 INFO:teuthology.orchestra.run.smithi122.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-11T08:01:24.385 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-11T08:01:24.391 INFO:teuthology.orchestra.run.smithi122.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-11T08:01:24.391 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stdout inactive 2024-09-11T08:01:24.398 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stdout enabled 2024-09-11T08:01:24.405 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stdout active 2024-09-11T08:01:24.405 INFO:teuthology.orchestra.run.smithi122.stdout:Unit chronyd.service is enabled and running 2024-09-11T08:01:24.405 INFO:teuthology.orchestra.run.smithi122.stdout:Repeating the final host check... 2024-09-11T08:01:24.427 INFO:teuthology.orchestra.run.smithi122.stdout:/bin/podman: stdout 5.2.2 2024-09-11T08:01:24.427 INFO:teuthology.orchestra.run.smithi122.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-11T08:01:24.427 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl is present 2024-09-11T08:01:24.427 INFO:teuthology.orchestra.run.smithi122.stdout:lvcreate is present 2024-09-11T08:01:24.434 INFO:teuthology.orchestra.run.smithi122.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-11T08:01:24.434 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-11T08:01:24.440 INFO:teuthology.orchestra.run.smithi122.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-11T08:01:24.440 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stdout inactive 2024-09-11T08:01:24.447 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stdout enabled 2024-09-11T08:01:24.454 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stdout active 2024-09-11T08:01:24.454 INFO:teuthology.orchestra.run.smithi122.stdout:Unit chronyd.service is enabled and running 2024-09-11T08:01:24.454 INFO:teuthology.orchestra.run.smithi122.stdout:Host looks OK 2024-09-11T08:01:24.454 INFO:teuthology.orchestra.run.smithi122.stdout:Cluster fsid: f1d75850-7013-11ef-bcea-c7b262605968 2024-09-11T08:01:24.454 INFO:teuthology.orchestra.run.smithi122.stdout:Acquiring lock 140406799708944 on /run/cephadm/f1d75850-7013-11ef-bcea-c7b262605968.lock 2024-09-11T08:01:24.454 INFO:teuthology.orchestra.run.smithi122.stdout:Lock 140406799708944 acquired on /run/cephadm/f1d75850-7013-11ef-bcea-c7b262605968.lock 2024-09-11T08:01:24.455 INFO:teuthology.orchestra.run.smithi122.stdout:Verifying IP 172.21.15.122 port 3300 ... 2024-09-11T08:01:24.455 INFO:teuthology.orchestra.run.smithi122.stdout:Verifying IP 172.21.15.122 port 6789 ... 2024-09-11T08:01:24.455 INFO:teuthology.orchestra.run.smithi122.stdout:Base mon IP(s) is [172.21.15.122:3300, 172.21.15.122:6789], mon addrv is [v2:172.21.15.122:3300,v1:172.21.15.122:6789] 2024-09-11T08:01:24.458 INFO:teuthology.orchestra.run.smithi122.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.122 metric 100 2024-09-11T08:01:24.458 INFO:teuthology.orchestra.run.smithi122.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.122 metric 100 2024-09-11T08:01:24.461 INFO:teuthology.orchestra.run.smithi122.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-11T08:01:24.461 INFO:teuthology.orchestra.run.smithi122.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-09-11T08:01:24.461 INFO:teuthology.orchestra.run.smithi122.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-09-11T08:01:24.464 INFO:teuthology.orchestra.run.smithi122.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-11T08:01:24.464 INFO:teuthology.orchestra.run.smithi122.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-11T08:01:24.464 INFO:teuthology.orchestra.run.smithi122.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-11T08:01:24.464 INFO:teuthology.orchestra.run.smithi122.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-09-11T08:01:24.464 INFO:teuthology.orchestra.run.smithi122.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:d515/64 scope link noprefixroute 2024-09-11T08:01:24.464 INFO:teuthology.orchestra.run.smithi122.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-11T08:01:24.466 INFO:teuthology.orchestra.run.smithi122.stdout:Mon IP `172.21.15.122` is in CIDR network `172.21.0.0/20` 2024-09-11T08:01:24.466 INFO:teuthology.orchestra.run.smithi122.stdout:Mon IP `172.21.15.122` is in CIDR network `172.21.0.0/20` 2024-09-11T08:01:24.466 INFO:teuthology.orchestra.run.smithi122.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-11T08:01:24.466 INFO:teuthology.orchestra.run.smithi122.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-11T08:01:24.466 INFO:teuthology.orchestra.run.smithi122.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9... 2024-09-11T08:01:25.088 INFO:teuthology.orchestra.run.smithi122.stdout:/bin/podman: stdout 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b 2024-09-11T08:01:25.088 INFO:teuthology.orchestra.run.smithi122.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9... 2024-09-11T08:01:25.088 INFO:teuthology.orchestra.run.smithi122.stdout:/bin/podman: stderr Getting image source signatures 2024-09-11T08:01:25.089 INFO:teuthology.orchestra.run.smithi122.stdout:/bin/podman: stderr Copying blob sha256:0d2f264d63cec2f6fb0f85eed7b2c8c5b4db3ab9edd9d71a74c04647294a553f 2024-09-11T08:01:25.089 INFO:teuthology.orchestra.run.smithi122.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-11T08:01:25.089 INFO:teuthology.orchestra.run.smithi122.stdout:/bin/podman: stderr Copying config sha256:4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b 2024-09-11T08:01:25.089 INFO:teuthology.orchestra.run.smithi122.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-11T08:01:26.017 INFO:teuthology.orchestra.run.smithi122.stdout:ceph: stdout ceph version 17.2.7-1623-gd0e6828a (d0e6828a2016d48cf25ad84064e50742bb1c39b9) quincy (stable) 2024-09-11T08:01:26.017 INFO:teuthology.orchestra.run.smithi122.stdout:Ceph version: ceph version 17.2.7-1623-gd0e6828a (d0e6828a2016d48cf25ad84064e50742bb1c39b9) quincy (stable) 2024-09-11T08:01:26.017 INFO:teuthology.orchestra.run.smithi122.stdout:Extracting ceph user uid/gid from container image... 2024-09-11T08:01:26.882 INFO:teuthology.orchestra.run.smithi122.stdout:stat: stdout 167 167 2024-09-11T08:01:26.882 INFO:teuthology.orchestra.run.smithi122.stdout:Creating initial keys... 2024-09-11T08:01:27.921 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph-authtool: stdout AQBXTuFmC7QVDRAA4olrgaAzdJczTB8DDvgiRA== 2024-09-11T08:01:28.810 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph-authtool: stdout AQBYTuFmwxTgDBAAN/kyNGlxzOhsULTz/c9LSA== 2024-09-11T08:01:29.758 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph-authtool: stdout AQBZTuFml29WBxAAvLofKoNFoe1uzS337O1tEA== 2024-09-11T08:01:29.759 INFO:teuthology.orchestra.run.smithi122.stdout:Creating initial monmap... 2024-09-11T08:01:30.698 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-11T08:01:30.698 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = octopus 2024-09-11T08:01:30.698 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to f1d75850-7013-11ef-bcea-c7b262605968 2024-09-11T08:01:30.698 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-11T08:01:30.699 INFO:teuthology.orchestra.run.smithi122.stdout:monmaptool for a [v2:172.21.15.122:3300,v1:172.21.15.122:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-11T08:01:30.699 INFO:teuthology.orchestra.run.smithi122.stdout:setting min_mon_release = octopus 2024-09-11T08:01:30.699 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/monmaptool: set fsid to f1d75850-7013-11ef-bcea-c7b262605968 2024-09-11T08:01:30.699 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-11T08:01:30.699 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T08:01:30.699 INFO:teuthology.orchestra.run.smithi122.stdout:Creating mon... 2024-09-11T08:01:31.745 INFO:teuthology.orchestra.run.smithi122.stdout:create mon.a on 2024-09-11T08:01:31.943 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-11T08:01:32.083 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-11T08:01:32.230 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-f1d75850-7013-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-f1d75850-7013-11ef-bcea-c7b262605968.target. 2024-09-11T08:01:32.230 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-f1d75850-7013-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-f1d75850-7013-11ef-bcea-c7b262605968.target. 2024-09-11T08:01:32.393 INFO:teuthology.orchestra.run.smithi122.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-f1d75850-7013-11ef-bcea-c7b262605968@mon.a 2024-09-11T08:01:32.393 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stderr Failed to reset failed state of unit ceph-f1d75850-7013-11ef-bcea-c7b262605968@mon.a.service: Unit ceph-f1d75850-7013-11ef-bcea-c7b262605968@mon.a.service not loaded. 2024-09-11T08:01:32.555 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-f1d75850-7013-11ef-bcea-c7b262605968.target.wants/ceph-f1d75850-7013-11ef-bcea-c7b262605968@mon.a.service → /etc/systemd/system/ceph-f1d75850-7013-11ef-bcea-c7b262605968@.service. 2024-09-11T08:01:33.106 INFO:teuthology.orchestra.run.smithi122.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-11T08:01:33.106 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stdout disabled 2024-09-11T08:01:33.114 INFO:teuthology.orchestra.run.smithi122.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-11T08:01:33.114 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stdout inactive 2024-09-11T08:01:33.114 INFO:teuthology.orchestra.run.smithi122.stdout:firewalld.service is not enabled 2024-09-11T08:01:33.114 INFO:teuthology.orchestra.run.smithi122.stdout:Not possible to enable service . firewalld.service is not available 2024-09-11T08:01:33.115 INFO:teuthology.orchestra.run.smithi122.stdout:Waiting for mon to start... 2024-09-11T08:01:33.115 INFO:teuthology.orchestra.run.smithi122.stdout:Waiting for mon... 2024-09-11T08:01:34.207 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout cluster: 2024-09-11T08:01:34.208 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout id: f1d75850-7013-11ef-bcea-c7b262605968 2024-09-11T08:01:34.208 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-11T08:01:34.208 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout 2024-09-11T08:01:34.208 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout services: 2024-09-11T08:01:34.208 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.628994s) 2024-09-11T08:01:34.208 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-11T08:01:34.208 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-11T08:01:34.208 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout 2024-09-11T08:01:34.208 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout data: 2024-09-11T08:01:34.208 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-11T08:01:34.208 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-11T08:01:34.209 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-11T08:01:34.209 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout pgs: 2024-09-11T08:01:34.209 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout 2024-09-11T08:01:34.209 INFO:teuthology.orchestra.run.smithi122.stdout:mon is available 2024-09-11T08:01:34.209 INFO:teuthology.orchestra.run.smithi122.stdout:Assimilating anything we can from ceph.conf... 2024-09-11T08:01:35.293 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:35 smithi122 ceph-mon[28620]: from='client.? 172.21.15.122:0/2105455139' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-11T08:01:35.294 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout 2024-09-11T08:01:35.297 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout [global] 2024-09-11T08:01:35.297 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout fsid = f1d75850-7013-11ef-bcea-c7b262605968 2024-09-11T08:01:35.297 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-11T08:01:35.297 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.122:3300,v1:172.21.15.122:6789] 2024-09-11T08:01:35.297 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-11T08:01:35.297 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-11T08:01:35.297 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-11T08:01:35.297 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-11T08:01:35.297 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout 2024-09-11T08:01:35.297 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-11T08:01:35.297 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-11T08:01:35.298 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout 2024-09-11T08:01:35.298 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout [osd] 2024-09-11T08:01:35.298 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-11T08:01:35.298 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-11T08:01:35.298 INFO:teuthology.orchestra.run.smithi122.stdout:Generating new minimal ceph.conf... 2024-09-11T08:01:36.307 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:36 smithi122 ceph-mon[28620]: from='client.? 172.21.15.122:0/675210492' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:01:36.310 INFO:teuthology.orchestra.run.smithi122.stdout:Restarting the monitor... 2024-09-11T08:01:36.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:36 smithi122 systemd[1]: Stopping Ceph mon.a for f1d75850-7013-11ef-bcea-c7b262605968... 2024-09-11T08:01:36.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:36 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mon-a[28616]: 2024-09-11T08:01:36.551+0000 7fe8fa269640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T08:01:36.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:36 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mon-a[28616]: 2024-09-11T08:01:36.551+0000 7fe8fa269640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-11T08:01:36.993 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:36 smithi122 podman[28869]: 2024-09-11 08:01:36.735804187 +0000 UTC m=+0.266556894 container died 9c0d58387dd83219c50b2048e8644072c67b580a649d89c74f391376482f7be3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-mon-a, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD) 2024-09-11T08:01:37.352 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 podman[28869]: 2024-09-11 08:01:37.092943193 +0000 UTC m=+0.623695896 container cleanup 9c0d58387dd83219c50b2048e8644072c67b580a649d89c74f391376482f7be3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-mon-a, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-11T08:01:37.353 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 bash[28869]: ceph-f1d75850-7013-11ef-bcea-c7b262605968-mon-a 2024-09-11T08:01:37.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 podman[28881]: 2024-09-11 08:01:37.352228821 +0000 UTC m=+0.613247082 container remove 9c0d58387dd83219c50b2048e8644072c67b580a649d89c74f391376482f7be3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-mon-a, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-11T08:01:37.711 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 systemd[1]: ceph-f1d75850-7013-11ef-bcea-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-11T08:01:37.712 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 systemd[1]: Stopped Ceph mon.a for f1d75850-7013-11ef-bcea-c7b262605968. 2024-09-11T08:01:37.712 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 systemd[1]: Starting Ceph mon.a for f1d75850-7013-11ef-bcea-c7b262605968... 2024-09-11T08:01:37.986 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 podman[28973]: 2024-09-11 08:01:37.711162244 +0000 UTC m=+0.106209637 container create b720c3646b61a425df5c73d63fcebd34a3f9998e9a2a85892a54334c1f8dd520 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-mon-a, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True) 2024-09-11T08:01:37.986 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 podman[28973]: 2024-09-11 08:01:37.620778189 +0000 UTC m=+0.015825589 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 2024-09-11T08:01:37.987 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 podman[28973]: 2024-09-11 08:01:37.905316492 +0000 UTC m=+0.300363886 container init b720c3646b61a425df5c73d63fcebd34a3f9998e9a2a85892a54334c1f8dd520 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-mon-a, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True) 2024-09-11T08:01:37.987 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 podman[28973]: 2024-09-11 08:01:37.908354434 +0000 UTC m=+0.303401828 container start b720c3646b61a425df5c73d63fcebd34a3f9998e9a2a85892a54334c1f8dd520 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-mon-a, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-11T08:01:37.987 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: set uid:gid to 167:167 (ceph:ceph) 2024-09-11T08:01:37.987 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: ceph version 17.2.7-1623-gd0e6828a (d0e6828a2016d48cf25ad84064e50742bb1c39b9) quincy (stable), process ceph-mon, pid 2 2024-09-11T08:01:37.987 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: pidfile_write: ignore empty --pid-file 2024-09-11T08:01:37.987 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: load: jerasure load: lrc 2024-09-11T08:01:37.987 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: RocksDB version: 6.15.5 2024-09-11T08:01:37.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-09-11T08:01:37.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Compile date Sep 10 2024 2024-09-11T08:01:37.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: DB SUMMARY 2024-09-11T08:01:37.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: DB Session ID: PGCLMHVSZAG2H930VWXO 2024-09-11T08:01:37.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: CURRENT file: CURRENT 2024-09-11T08:01:37.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: IDENTITY file: IDENTITY 2024-09-11T08:01:37.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: MANIFEST file: MANIFEST-000009 size: 131 Bytes 2024-09-11T08:01:37.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-11T08:01:37.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000010.log size: 83282 ; 2024-09-11T08:01:37.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.error_if_exists: 0 2024-09-11T08:01:37.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.create_if_missing: 0 2024-09-11T08:01:37.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.paranoid_checks: 1 2024-09-11T08:01:37.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-11T08:01:37.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.env: 0x55e8ad21d440 2024-09-11T08:01:37.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.fs: Posix File System 2024-09-11T08:01:37.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.info_log: 0x55e8af152ec0 2024-09-11T08:01:37.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.max_file_opening_threads: 16 2024-09-11T08:01:37.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.statistics: (nil) 2024-09-11T08:01:37.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.use_fsync: 0 2024-09-11T08:01:37.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.max_log_file_size: 0 2024-09-11T08:01:37.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-11T08:01:37.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-11T08:01:37.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.keep_log_file_num: 1000 2024-09-11T08:01:37.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.recycle_log_file_num: 0 2024-09-11T08:01:37.990 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.allow_fallocate: 1 2024-09-11T08:01:37.990 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.allow_mmap_reads: 0 2024-09-11T08:01:37.990 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.allow_mmap_writes: 0 2024-09-11T08:01:37.990 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.use_direct_reads: 0 2024-09-11T08:01:37.990 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-11T08:01:37.990 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.create_missing_column_families: 0 2024-09-11T08:01:37.990 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.db_log_dir: 2024-09-11T08:01:37.990 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-a/store.db 2024-09-11T08:01:37.990 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-11T08:01:37.990 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-11T08:01:37.990 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-11T08:01:37.990 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-11T08:01:37.990 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-11T08:01:37.990 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-11T08:01:37.991 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.advise_random_on_open: 1 2024-09-11T08:01:37.991 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.db_write_buffer_size: 0 2024-09-11T08:01:37.991 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.write_buffer_manager: 0x55e8af210780 2024-09-11T08:01:37.991 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-11T08:01:37.991 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-09-11T08:01:37.991 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-11T08:01:37.991 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-11T08:01:37.991 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.rate_limiter: (nil) 2024-09-11T08:01:37.991 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-11T08:01:37.991 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.wal_recovery_mode: 2 2024-09-11T08:01:37.991 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.enable_thread_tracking: 0 2024-09-11T08:01:37.991 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.enable_pipelined_write: 0 2024-09-11T08:01:37.991 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.unordered_write: 0 2024-09-11T08:01:37.992 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-11T08:01:37.992 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-11T08:01:37.992 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-11T08:01:37.992 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-11T08:01:37.992 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.row_cache: None 2024-09-11T08:01:37.992 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.wal_filter: None 2024-09-11T08:01:37.992 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-11T08:01:37.992 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.allow_ingest_behind: 0 2024-09-11T08:01:37.992 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.preserve_deletes: 0 2024-09-11T08:01:37.992 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.two_write_queues: 0 2024-09-11T08:01:37.992 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.manual_wal_flush: 0 2024-09-11T08:01:37.992 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.atomic_flush: 0 2024-09-11T08:01:37.992 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-11T08:01:37.993 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-11T08:01:37.993 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-11T08:01:37.993 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.log_readahead_size: 0 2024-09-11T08:01:37.993 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-11T08:01:37.993 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.best_efforts_recovery: 0 2024-09-11T08:01:37.993 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-11T08:01:37.993 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-11T08:01:37.993 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.allow_data_in_errors: 0 2024-09-11T08:01:37.993 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.db_host_id: __hostname__ 2024-09-11T08:01:37.994 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.max_background_jobs: 2 2024-09-11T08:01:37.994 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.max_background_compactions: -1 2024-09-11T08:01:37.994 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.max_subcompactions: 1 2024-09-11T08:01:37.994 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-11T08:01:37.994 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-11T08:01:37.994 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-11T08:01:37.994 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.max_total_wal_size: 0 2024-09-11T08:01:37.994 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-11T08:01:37.994 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-11T08:01:37.994 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-11T08:01:37.995 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-11T08:01:37.995 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.max_open_files: -1 2024-09-11T08:01:37.995 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.bytes_per_sync: 0 2024-09-11T08:01:37.995 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-11T08:01:37.995 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-11T08:01:37.995 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.compaction_readahead_size: 0 2024-09-11T08:01:37.995 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.max_background_flushes: -1 2024-09-11T08:01:37.995 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Compression algorithms supported: 2024-09-11T08:01:37.995 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: kZSTD supported: 0 2024-09-11T08:01:37.996 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: kXpressCompression supported: 0 2024-09-11T08:01:37.996 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: kBZip2Compression supported: 0 2024-09-11T08:01:37.996 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-11T08:01:37.996 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: kLZ4Compression supported: 1 2024-09-11T08:01:37.996 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: kZlibCompression supported: 1 2024-09-11T08:01:37.996 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: kLZ4HCCompression supported: 1 2024-09-11T08:01:37.996 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: kSnappyCompression supported: 1 2024-09-11T08:01:37.996 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-11T08:01:37.996 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000009 2024-09-11T08:01:37.996 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2024-09-11T08:01:37.996 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-11T08:01:37.996 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.merge_operator: 2024-09-11T08:01:37.997 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.compaction_filter: None 2024-09-11T08:01:37.997 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.compaction_filter_factory: None 2024-09-11T08:01:37.997 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.sst_partitioner_factory: None 2024-09-11T08:01:37.997 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-11T08:01:37.997 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-11T08:01:37.997 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55e8af0f9a38) 2024-09-11T08:01:37.997 INFO:journalctl@ceph.mon.a.smithi122.stdout: cache_index_and_filter_blocks: 1 2024-09-11T08:01:37.997 INFO:journalctl@ceph.mon.a.smithi122.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-11T08:01:37.997 INFO:journalctl@ceph.mon.a.smithi122.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-11T08:01:37.997 INFO:journalctl@ceph.mon.a.smithi122.stdout: pin_top_level_index_and_filter: 1 2024-09-11T08:01:37.997 INFO:journalctl@ceph.mon.a.smithi122.stdout: index_type: 0 2024-09-11T08:01:37.997 INFO:journalctl@ceph.mon.a.smithi122.stdout: data_block_index_type: 0 2024-09-11T08:01:37.997 INFO:journalctl@ceph.mon.a.smithi122.stdout: index_shortening: 1 2024-09-11T08:01:37.998 INFO:journalctl@ceph.mon.a.smithi122.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-11T08:01:37.998 INFO:journalctl@ceph.mon.a.smithi122.stdout: hash_index_allow_collision: 1 2024-09-11T08:01:37.998 INFO:journalctl@ceph.mon.a.smithi122.stdout: checksum: 1 2024-09-11T08:01:37.998 INFO:journalctl@ceph.mon.a.smithi122.stdout: no_block_cache: 0 2024-09-11T08:01:37.998 INFO:journalctl@ceph.mon.a.smithi122.stdout: block_cache: 0x55e8af12f1f0 2024-09-11T08:01:37.998 INFO:journalctl@ceph.mon.a.smithi122.stdout: block_cache_name: BinnedLRUCache 2024-09-11T08:01:37.998 INFO:journalctl@ceph.mon.a.smithi122.stdout: block_cache_options: 2024-09-11T08:01:37.998 INFO:journalctl@ceph.mon.a.smithi122.stdout: capacity : 536870912 2024-09-11T08:01:37.998 INFO:journalctl@ceph.mon.a.smithi122.stdout: num_shard_bits : 4 2024-09-11T08:01:37.998 INFO:journalctl@ceph.mon.a.smithi122.stdout: strict_capacity_limit : 0 2024-09-11T08:01:37.998 INFO:journalctl@ceph.mon.a.smithi122.stdout: high_pri_pool_ratio: 0.000 2024-09-11T08:01:37.998 INFO:journalctl@ceph.mon.a.smithi122.stdout: block_cache_compressed: (nil) 2024-09-11T08:01:37.998 INFO:journalctl@ceph.mon.a.smithi122.stdout: persistent_cache: (nil) 2024-09-11T08:01:37.998 INFO:journalctl@ceph.mon.a.smithi122.stdout: block_size: 4096 2024-09-11T08:01:37.998 INFO:journalctl@ceph.mon.a.smithi122.stdout: block_size_deviation: 10 2024-09-11T08:01:37.998 INFO:journalctl@ceph.mon.a.smithi122.stdout: block_restart_interval: 16 2024-09-11T08:01:37.999 INFO:journalctl@ceph.mon.a.smithi122.stdout: index_block_restart_interval: 1 2024-09-11T08:01:37.999 INFO:journalctl@ceph.mon.a.smithi122.stdout: metadata_block_size: 4096 2024-09-11T08:01:37.999 INFO:journalctl@ceph.mon.a.smithi122.stdout: partition_filters: 0 2024-09-11T08:01:37.999 INFO:journalctl@ceph.mon.a.smithi122.stdout: use_delta_encoding: 1 2024-09-11T08:01:37.999 INFO:journalctl@ceph.mon.a.smithi122.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2024-09-11T08:01:37.999 INFO:journalctl@ceph.mon.a.smithi122.stdout: whole_key_filtering: 1 2024-09-11T08:01:37.999 INFO:journalctl@ceph.mon.a.smithi122.stdout: verify_compression: 0 2024-09-11T08:01:37.999 INFO:journalctl@ceph.mon.a.smithi122.stdout: read_amp_bytes_per_bit: 0 2024-09-11T08:01:37.999 INFO:journalctl@ceph.mon.a.smithi122.stdout: format_version: 4 2024-09-11T08:01:37.999 INFO:journalctl@ceph.mon.a.smithi122.stdout: enable_index_compression: 1 2024-09-11T08:01:37.999 INFO:journalctl@ceph.mon.a.smithi122.stdout: block_align: 0 2024-09-11T08:01:37.999 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.write_buffer_size: 33554432 2024-09-11T08:01:37.999 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.max_write_buffer_number: 2 2024-09-11T08:01:37.999 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.compression: NoCompression 2024-09-11T08:01:38.000 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.bottommost_compression: Disabled 2024-09-11T08:01:38.000 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.prefix_extractor: nullptr 2024-09-11T08:01:38.000 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-11T08:01:38.000 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.num_levels: 7 2024-09-11T08:01:38.000 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-11T08:01:38.000 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-11T08:01:38.000 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-11T08:01:38.000 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-11T08:01:38.000 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-11T08:01:38.000 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-11T08:01:38.000 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-11T08:01:38.000 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-11T08:01:38.000 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-11T08:01:38.000 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-11T08:01:38.001 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-11T08:01:38.001 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.compression_opts.level: 32767 2024-09-11T08:01:38.001 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.compression_opts.strategy: 0 2024-09-11T08:01:38.001 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-11T08:01:38.001 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-11T08:01:38.001 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-11T08:01:38.001 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.compression_opts.enabled: false 2024-09-11T08:01:38.001 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-11T08:01:38.001 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-11T08:01:38.001 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-11T08:01:38.001 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.target_file_size_base: 67108864 2024-09-11T08:01:38.001 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-11T08:01:38.001 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-11T08:01:38.002 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-11T08:01:38.002 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-11T08:01:38.002 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-11T08:01:38.003 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-11T08:01:38.003 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-11T08:01:38.003 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-11T08:01:38.003 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-11T08:01:38.003 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-11T08:01:38.003 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-11T08:01:38.003 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-11T08:01:38.003 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-11T08:01:38.003 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.arena_block_size: 4194304 2024-09-11T08:01:38.003 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-11T08:01:38.003 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-11T08:01:38.003 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-09-11T08:01:38.003 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.disable_auto_compactions: 0 2024-09-11T08:01:38.004 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-11T08:01:38.004 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-11T08:01:38.004 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-11T08:01:38.004 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-11T08:01:38.004 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-11T08:01:38.004 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-11T08:01:38.004 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-11T08:01:38.004 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-11T08:01:38.004 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-11T08:01:38.004 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-11T08:01:38.004 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-11T08:01:38.004 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.inplace_update_support: 0 2024-09-11T08:01:38.004 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-11T08:01:38.004 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-11T08:01:38.005 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-11T08:01:38.005 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-11T08:01:38.005 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.bloom_locality: 0 2024-09-11T08:01:38.005 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.max_successive_merges: 0 2024-09-11T08:01:38.005 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-11T08:01:38.005 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.paranoid_file_checks: 0 2024-09-11T08:01:38.005 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.force_consistency_checks: 1 2024-09-11T08:01:38.005 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.report_bg_io_stats: 0 2024-09-11T08:01:38.005 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.ttl: 2592000 2024-09-11T08:01:38.005 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-11T08:01:38.005 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.enable_blob_files: false 2024-09-11T08:01:38.005 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.min_blob_size: 0 2024-09-11T08:01:38.006 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.blob_file_size: 268435456 2024-09-11T08:01:38.006 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-11T08:01:38.006 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-11T08:01:38.006 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-11T08:01:38.006 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000009 succeeded,manifest_file_number is 9, next_file_number is 11, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-11T08:01:38.006 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 5 2024-09-11T08:01:38.006 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: [db/version_set.cc:4082] Creating manifest 13 2024-09-11T08:01:38.006 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726041697931859, "job": 1, "event": "recovery_started", "wal_files": [10]} 2024-09-11T08:01:38.006 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #10 mode 2 2024-09-11T08:01:38.006 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: [table/block_based/filter_policy.cc:991] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-09-11T08:01:38.006 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726041697932581, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 14, "file_size": 80006, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 78241, "index_size": 227, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 581, "raw_key_size": 9858, "raw_average_key_size": 47, "raw_value_size": 72562, "raw_average_value_size": 347, "num_data_blocks": 10, "num_entries": 209, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1726041697, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "4fe45251-624a-49ba-b1dc-9351263a88fa", "db_session_id": "PGCLMHVSZAG2H930VWXO"}} 2024-09-11T08:01:38.006 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: [db/version_set.cc:4082] Creating manifest 15 2024-09-11T08:01:38.006 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726041697932914, "job": 1, "event": "recovery_finished"} 2024-09-11T08:01:38.006 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000010.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-11T08:01:38.006 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x55e8af162700 2024-09-11T08:01:38.007 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: DB pointer 0x55e8af132000 2024-09-11T08:01:38.007 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2024-09-11T08:01:38.007 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: rocksdb: [db/db_impl/db_impl.cc:903] 2024-09-11T08:01:38.007 INFO:journalctl@ceph.mon.a.smithi122.stdout: ** DB Stats ** 2024-09-11T08:01:38.007 INFO:journalctl@ceph.mon.a.smithi122.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-11T08:01:38.007 INFO:journalctl@ceph.mon.a.smithi122.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-11T08:01:38.007 INFO:journalctl@ceph.mon.a.smithi122.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-11T08:01:38.007 INFO:journalctl@ceph.mon.a.smithi122.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-11T08:01:38.007 INFO:journalctl@ceph.mon.a.smithi122.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-11T08:01:38.007 INFO:journalctl@ceph.mon.a.smithi122.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-09-11T08:01:38.007 INFO:journalctl@ceph.mon.a.smithi122.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-11T08:01:38.007 INFO:journalctl@ceph.mon.a.smithi122.stdout: 2024-09-11T08:01:38.007 INFO:journalctl@ceph.mon.a.smithi122.stdout: ** Compaction Stats [default] ** 2024-09-11T08:01:38.007 INFO:journalctl@ceph.mon.a.smithi122.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-11T08:01:38.008 INFO:journalctl@ceph.mon.a.smithi122.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-11T08:01:38.008 INFO:journalctl@ceph.mon.a.smithi122.stdout: L0 2/0 79.92 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 155.7 0.00 0.00 1 0.000 0 0 2024-09-11T08:01:38.008 INFO:journalctl@ceph.mon.a.smithi122.stdout: Sum 2/0 79.92 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 155.7 0.00 0.00 1 0.000 0 0 2024-09-11T08:01:38.008 INFO:journalctl@ceph.mon.a.smithi122.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 155.7 0.00 0.00 1 0.000 0 0 2024-09-11T08:01:38.008 INFO:journalctl@ceph.mon.a.smithi122.stdout: 2024-09-11T08:01:38.008 INFO:journalctl@ceph.mon.a.smithi122.stdout: ** Compaction Stats [default] ** 2024-09-11T08:01:38.008 INFO:journalctl@ceph.mon.a.smithi122.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-11T08:01:38.008 INFO:journalctl@ceph.mon.a.smithi122.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-11T08:01:38.008 INFO:journalctl@ceph.mon.a.smithi122.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 155.7 0.00 0.00 1 0.000 0 0 2024-09-11T08:01:38.008 INFO:journalctl@ceph.mon.a.smithi122.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-11T08:01:38.008 INFO:journalctl@ceph.mon.a.smithi122.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-11T08:01:38.008 INFO:journalctl@ceph.mon.a.smithi122.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-11T08:01:38.008 INFO:journalctl@ceph.mon.a.smithi122.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-11T08:01:38.008 INFO:journalctl@ceph.mon.a.smithi122.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-11T08:01:38.009 INFO:journalctl@ceph.mon.a.smithi122.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-11T08:01:38.009 INFO:journalctl@ceph.mon.a.smithi122.stdout: Cumulative compaction: 0.00 GB write, 25.32 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T08:01:38.009 INFO:journalctl@ceph.mon.a.smithi122.stdout: Interval compaction: 0.00 GB write, 25.32 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T08:01:38.009 INFO:journalctl@ceph.mon.a.smithi122.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-11T08:01:38.009 INFO:journalctl@ceph.mon.a.smithi122.stdout: 2024-09-11T08:01:38.009 INFO:journalctl@ceph.mon.a.smithi122.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-11T08:01:38.009 INFO:journalctl@ceph.mon.a.smithi122.stdout: 2024-09-11T08:01:38.009 INFO:journalctl@ceph.mon.a.smithi122.stdout: ** Compaction Stats [default] ** 2024-09-11T08:01:38.009 INFO:journalctl@ceph.mon.a.smithi122.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-11T08:01:38.009 INFO:journalctl@ceph.mon.a.smithi122.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-11T08:01:38.009 INFO:journalctl@ceph.mon.a.smithi122.stdout: L0 2/0 79.92 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 155.7 0.00 0.00 1 0.000 0 0 2024-09-11T08:01:38.009 INFO:journalctl@ceph.mon.a.smithi122.stdout: Sum 2/0 79.92 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 155.7 0.00 0.00 1 0.000 0 0 2024-09-11T08:01:38.009 INFO:journalctl@ceph.mon.a.smithi122.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-09-11T08:01:38.010 INFO:journalctl@ceph.mon.a.smithi122.stdout: 2024-09-11T08:01:38.010 INFO:journalctl@ceph.mon.a.smithi122.stdout: ** Compaction Stats [default] ** 2024-09-11T08:01:38.010 INFO:journalctl@ceph.mon.a.smithi122.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-11T08:01:38.010 INFO:journalctl@ceph.mon.a.smithi122.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-11T08:01:38.010 INFO:journalctl@ceph.mon.a.smithi122.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 155.7 0.00 0.00 1 0.000 0 0 2024-09-11T08:01:38.010 INFO:journalctl@ceph.mon.a.smithi122.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-11T08:01:38.010 INFO:journalctl@ceph.mon.a.smithi122.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-11T08:01:38.010 INFO:journalctl@ceph.mon.a.smithi122.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-11T08:01:38.010 INFO:journalctl@ceph.mon.a.smithi122.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-11T08:01:38.010 INFO:journalctl@ceph.mon.a.smithi122.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-11T08:01:38.010 INFO:journalctl@ceph.mon.a.smithi122.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-11T08:01:38.010 INFO:journalctl@ceph.mon.a.smithi122.stdout: Cumulative compaction: 0.00 GB write, 25.05 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T08:01:38.010 INFO:journalctl@ceph.mon.a.smithi122.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T08:01:38.010 INFO:journalctl@ceph.mon.a.smithi122.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-11T08:01:38.010 INFO:journalctl@ceph.mon.a.smithi122.stdout: 2024-09-11T08:01:38.011 INFO:journalctl@ceph.mon.a.smithi122.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-11T08:01:38.011 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: starting mon.a rank 0 at public addrs [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] at bind addrs [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid f1d75850-7013-11ef-bcea-c7b262605968 2024-09-11T08:01:38.011 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: mon.a@-1(???) e1 preinit fsid f1d75850-7013-11ef-bcea-c7b262605968 2024-09-11T08:01:38.011 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: mon.a@-1(???).mds e1 new map 2024-09-11T08:01:38.011 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: mon.a@-1(???).mds e1 print_map 2024-09-11T08:01:38.011 INFO:journalctl@ceph.mon.a.smithi122.stdout: e1 2024-09-11T08:01:38.011 INFO:journalctl@ceph.mon.a.smithi122.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-11T08:01:38.011 INFO:journalctl@ceph.mon.a.smithi122.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-11T08:01:38.011 INFO:journalctl@ceph.mon.a.smithi122.stdout: legacy client fscid: -1 2024-09-11T08:01:38.011 INFO:journalctl@ceph.mon.a.smithi122.stdout: 2024-09-11T08:01:38.011 INFO:journalctl@ceph.mon.a.smithi122.stdout: No filesystems configured 2024-09-11T08:01:38.011 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-11T08:01:38.011 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T08:01:38.011 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T08:01:38.012 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T08:01:38.012 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-11T08:01:38.012 INFO:teuthology.orchestra.run.smithi122.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-11T08:01:38.300 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 bash[28973]: b720c3646b61a425df5c73d63fcebd34a3f9998e9a2a85892a54334c1f8dd520 2024-09-11T08:01:38.300 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 systemd[1]: Started Ceph mon.a for f1d75850-7013-11ef-bcea-c7b262605968. 2024-09-11T08:01:38.301 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-11T08:01:38.301 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: monmap e1: 1 mons at {a=[v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-11T08:01:38.301 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: fsmap 2024-09-11T08:01:38.301 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: osdmap e1: 0 total, 0 up, 0 in 2024-09-11T08:01:38.301 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:37 smithi122 ceph-mon[28992]: mgrmap e1: no daemons active 2024-09-11T08:01:38.990 INFO:teuthology.orchestra.run.smithi122.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-11T08:01:38.990 INFO:teuthology.orchestra.run.smithi122.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-11T08:01:38.990 INFO:teuthology.orchestra.run.smithi122.stdout:Creating mgr... 2024-09-11T08:01:38.991 INFO:teuthology.orchestra.run.smithi122.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-11T08:01:38.991 INFO:teuthology.orchestra.run.smithi122.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-11T08:01:39.167 INFO:teuthology.orchestra.run.smithi122.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-f1d75850-7013-11ef-bcea-c7b262605968@mgr.y 2024-09-11T08:01:39.167 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stderr Failed to reset failed state of unit ceph-f1d75850-7013-11ef-bcea-c7b262605968@mgr.y.service: Unit ceph-f1d75850-7013-11ef-bcea-c7b262605968@mgr.y.service not loaded. 2024-09-11T08:01:39.315 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-f1d75850-7013-11ef-bcea-c7b262605968.target.wants/ceph-f1d75850-7013-11ef-bcea-c7b262605968@mgr.y.service → /etc/systemd/system/ceph-f1d75850-7013-11ef-bcea-c7b262605968@.service. 2024-09-11T08:01:39.444 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:01:39 smithi122 systemd[1]: Starting Ceph mgr.y for f1d75850-7013-11ef-bcea-c7b262605968... 2024-09-11T08:01:39.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:39 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/3157023593' entity='client.admin' 2024-09-11T08:01:39.734 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:01:39 smithi122 podman[29202]: 2024-09-11 08:01:39.452500896 +0000 UTC m=+0.025106572 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 2024-09-11T08:01:39.734 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:01:39 smithi122 podman[29202]: 2024-09-11 08:01:39.554465654 +0000 UTC m=+0.127071324 container create e4267cade852a6cfbb3eb68ed1593a40cf129d6b161427850f95232f005cd093 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-11T08:01:40.682 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:01:40 smithi122 podman[29202]: 2024-09-11 08:01:40.424934585 +0000 UTC m=+0.997540252 container init e4267cade852a6cfbb3eb68ed1593a40cf129d6b161427850f95232f005cd093 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-11T08:01:40.683 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:01:40 smithi122 podman[29202]: 2024-09-11 08:01:40.428897874 +0000 UTC m=+1.001503542 container start e4267cade852a6cfbb3eb68ed1593a40cf129d6b161427850f95232f005cd093 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-11T08:01:40.683 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:01:40 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:01:40.553+0000 7fb947054ec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T08:01:40.683 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:01:40 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:01:40.609+0000 7fb947054ec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T08:01:40.683 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:01:40 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:01:40.674+0000 7fb947054ec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T08:01:40.932 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:01:40 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:01:40.797+0000 7fb947054ec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T08:01:40.933 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:01:40 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:01:40.932+0000 7fb947054ec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T08:01:41.462 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:01:41 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:01:41.192+0000 7fb947054ec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T08:01:41.463 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:01:41 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:01:41.255+0000 7fb947054ec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T08:01:41.463 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:01:41 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:01:41.419+0000 7fb947054ec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T08:01:41.477 INFO:teuthology.orchestra.run.smithi122.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-11T08:01:41.477 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stdout disabled 2024-09-11T08:01:41.485 INFO:teuthology.orchestra.run.smithi122.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-11T08:01:41.485 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stdout inactive 2024-09-11T08:01:41.485 INFO:teuthology.orchestra.run.smithi122.stdout:firewalld.service is not enabled 2024-09-11T08:01:41.485 INFO:teuthology.orchestra.run.smithi122.stdout:Not possible to enable service . firewalld.service is not available 2024-09-11T08:01:41.494 INFO:teuthology.orchestra.run.smithi122.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-11T08:01:41.494 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stdout disabled 2024-09-11T08:01:41.502 INFO:teuthology.orchestra.run.smithi122.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-11T08:01:41.502 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stdout inactive 2024-09-11T08:01:41.502 INFO:teuthology.orchestra.run.smithi122.stdout:firewalld.service is not enabled 2024-09-11T08:01:41.502 INFO:teuthology.orchestra.run.smithi122.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-11T08:01:41.502 INFO:teuthology.orchestra.run.smithi122.stdout:Waiting for mgr to start... 2024-09-11T08:01:41.502 INFO:teuthology.orchestra.run.smithi122.stdout:Waiting for mgr... 2024-09-11T08:01:41.734 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:01:41 smithi122 bash[29202]: e4267cade852a6cfbb3eb68ed1593a40cf129d6b161427850f95232f005cd093 2024-09-11T08:01:41.734 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:01:41 smithi122 systemd[1]: Started Ceph mgr.y for f1d75850-7013-11ef-bcea-c7b262605968. 2024-09-11T08:01:42.734 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:01:42 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:01:42.298+0000 7fb947054ec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T08:01:42.734 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:01:42 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:01:42.363+0000 7fb947054ec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T08:01:43.366 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:01:43 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:01:43.076+0000 7fb947054ec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T08:01:43.366 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:01:43 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:01:43.187+0000 7fb947054ec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T08:01:43.367 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:01:43 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:01:43.307+0000 7fb947054ec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T08:01:43.367 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:01:43 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:01:43.365+0000 7fb947054ec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T08:01:43.619 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:01:43 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:01:43.495+0000 7fb947054ec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T08:01:43.619 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:01:43 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:01:43.561+0000 7fb947054ec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T08:01:43.619 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:01:43 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:01:43.618+0000 7fb947054ec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T08:01:43.883 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:01:43 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:01:43.751+0000 7fb947054ec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T08:01:43.883 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:01:43 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:01:43.815+0000 7fb947054ec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T08:01:43.883 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:01:43 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:01:43.882+0000 7fb947054ec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T08:01:44.234 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:01:43 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:01:43.962+0000 7fb947054ec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T08:01:44.234 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:01:44 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:01:44.018+0000 7fb947054ec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T08:01:44.234 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:01:44 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:01:44.076+0000 7fb947054ec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T08:01:44.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:44 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/1495749651' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-11T08:01:45.234 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:01:44 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:01:44.835+0000 7fb947054ec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T08:01:45.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:45 smithi122 ceph-mon[28992]: Activating manager daemon y 2024-09-11T08:01:45.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:45 smithi122 ceph-mon[28992]: mgrmap e2: y(active, starting, since 0.00273784s) 2024-09-11T08:01:45.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:45 smithi122 ceph-mon[28992]: from='mgr.14100 172.21.15.122:0/3479165094' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T08:01:45.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:45 smithi122 ceph-mon[28992]: from='mgr.14100 172.21.15.122:0/3479165094' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T08:01:45.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:45 smithi122 ceph-mon[28992]: from='mgr.14100 172.21.15.122:0/3479165094' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T08:01:45.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:45 smithi122 ceph-mon[28992]: from='mgr.14100 172.21.15.122:0/3479165094' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:01:45.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:45 smithi122 ceph-mon[28992]: from='mgr.14100 172.21.15.122:0/3479165094' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T08:01:45.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:45 smithi122 ceph-mon[28992]: Manager daemon y is now available 2024-09-11T08:01:45.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:45 smithi122 ceph-mon[28992]: from='mgr.14100 172.21.15.122:0/3479165094' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-11T08:01:45.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:45 smithi122 ceph-mon[28992]: from='mgr.14100 172.21.15.122:0/3479165094' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-11T08:01:45.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:45 smithi122 ceph-mon[28992]: from='mgr.14100 172.21.15.122:0/3479165094' entity='mgr.y' 2024-09-11T08:01:45.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:45 smithi122 ceph-mon[28992]: from='mgr.14100 172.21.15.122:0/3479165094' entity='mgr.y' 2024-09-11T08:01:45.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:45 smithi122 ceph-mon[28992]: from='mgr.14100 172.21.15.122:0/3479165094' entity='mgr.y' 2024-09-11T08:01:47.234 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:46 smithi122 ceph-mon[28992]: mgrmap e3: y(active, since 1.00669s) 2024-09-11T08:01:48.123 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:47 smithi122 ceph-mon[28992]: mgrmap e4: y(active, since 2s) 2024-09-11T08:01:49.013 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout 2024-09-11T08:01:49.013 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout { 2024-09-11T08:01:49.013 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "fsid": "f1d75850-7013-11ef-bcea-c7b262605968", 2024-09-11T08:01:49.013 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "health": { 2024-09-11T08:01:49.013 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-11T08:01:49.013 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-11T08:01:49.013 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-11T08:01:49.013 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-09-11T08:01:49.013 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-11T08:01:49.013 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-11T08:01:49.014 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout 0 2024-09-11T08:01:49.014 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout ], 2024-09-11T08:01:49.014 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-11T08:01:49.014 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "a" 2024-09-11T08:01:49.014 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout ], 2024-09-11T08:01:49.014 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2024-09-11T08:01:49.014 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-11T08:01:49.014 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T08:01:49.014 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-09-11T08:01:49.014 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-11T08:01:49.014 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-09-11T08:01:49.014 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-11T08:01:49.014 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T08:01:49.014 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-11T08:01:49.015 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-11T08:01:49.015 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-11T08:01:49.015 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-11T08:01:49.015 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-11T08:01:49.015 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-11T08:01:49.018 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-09-11T08:01:49.018 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-11T08:01:49.018 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-11T08:01:49.018 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-11T08:01:49.018 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-11T08:01:49.018 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-11T08:01:49.019 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-11T08:01:49.019 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-11T08:01:49.019 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-11T08:01:49.019 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-11T08:01:49.019 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-09-11T08:01:49.019 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-11T08:01:49.019 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T08:01:49.019 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-11T08:01:49.019 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-11T08:01:49.019 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-09-11T08:01:49.019 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-11T08:01:49.019 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-11T08:01:49.019 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-11T08:01:49.020 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-11T08:01:49.020 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-11T08:01:49.020 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-11T08:01:49.020 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "restful" 2024-09-11T08:01:49.020 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout ], 2024-09-11T08:01:49.020 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T08:01:49.020 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-09-11T08:01:49.020 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-11T08:01:49.020 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T08:01:49.020 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "modified": "2024-09-11T08:01:33.023939+0000", 2024-09-11T08:01:49.020 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T08:01:49.020 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-09-11T08:01:49.020 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-11T08:01:49.020 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout } 2024-09-11T08:01:49.020 INFO:teuthology.orchestra.run.smithi122.stdout:mgr not available, waiting (1/15)... 2024-09-11T08:01:51.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:51 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/2546108496' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-11T08:01:52.201 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout 2024-09-11T08:01:52.201 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout { 2024-09-11T08:01:52.201 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "fsid": "f1d75850-7013-11ef-bcea-c7b262605968", 2024-09-11T08:01:52.201 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "health": { 2024-09-11T08:01:52.202 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-11T08:01:52.202 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-11T08:01:52.202 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-11T08:01:52.202 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-09-11T08:01:52.202 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-11T08:01:52.202 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-11T08:01:52.202 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout 0 2024-09-11T08:01:52.202 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout ], 2024-09-11T08:01:52.202 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-11T08:01:52.202 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "a" 2024-09-11T08:01:52.202 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout ], 2024-09-11T08:01:52.202 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "quorum_age": 13, 2024-09-11T08:01:52.202 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-11T08:01:52.203 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T08:01:52.203 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-09-11T08:01:52.203 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-11T08:01:52.203 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-09-11T08:01:52.203 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-11T08:01:52.203 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T08:01:52.203 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-11T08:01:52.203 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-11T08:01:52.203 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-11T08:01:52.203 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-11T08:01:52.203 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-11T08:01:52.203 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-11T08:01:52.207 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-09-11T08:01:52.207 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-11T08:01:52.207 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-11T08:01:52.207 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-11T08:01:52.207 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-11T08:01:52.207 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-11T08:01:52.207 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-11T08:01:52.207 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-11T08:01:52.207 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-11T08:01:52.207 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-11T08:01:52.208 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-09-11T08:01:52.208 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-11T08:01:52.208 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T08:01:52.208 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-11T08:01:52.208 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-11T08:01:52.208 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-09-11T08:01:52.208 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-11T08:01:52.208 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-11T08:01:52.208 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-11T08:01:52.208 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-11T08:01:52.208 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-11T08:01:52.208 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-11T08:01:52.208 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "restful" 2024-09-11T08:01:52.208 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout ], 2024-09-11T08:01:52.209 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T08:01:52.209 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-09-11T08:01:52.209 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-11T08:01:52.209 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T08:01:52.209 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "modified": "2024-09-11T08:01:33.023939+0000", 2024-09-11T08:01:52.209 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T08:01:52.209 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-09-11T08:01:52.209 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-11T08:01:52.209 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout } 2024-09-11T08:01:52.209 INFO:teuthology.orchestra.run.smithi122.stdout:mgr is available 2024-09-11T08:01:53.135 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:52 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/9788115' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-11T08:01:53.339 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout 2024-09-11T08:01:53.339 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout [global] 2024-09-11T08:01:53.339 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout fsid = f1d75850-7013-11ef-bcea-c7b262605968 2024-09-11T08:01:53.339 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-11T08:01:53.339 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-11T08:01:53.339 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-11T08:01:53.339 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-11T08:01:53.340 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-11T08:01:53.340 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout 2024-09-11T08:01:53.340 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-11T08:01:53.340 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-11T08:01:53.340 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout 2024-09-11T08:01:53.340 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout [osd] 2024-09-11T08:01:53.340 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-11T08:01:53.340 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-11T08:01:53.340 INFO:teuthology.orchestra.run.smithi122.stdout:Enabling cephadm module... 2024-09-11T08:01:54.027 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:54 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/3746144202' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-11T08:01:54.283 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:01:54 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: ignoring --setuser ceph since I am not root 2024-09-11T08:01:54.283 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:01:54 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: ignoring --setgroup ceph since I am not root 2024-09-11T08:01:54.283 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:01:54 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:01:54.157+0000 7f5e40ea9ec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T08:01:54.283 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:01:54 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:01:54.213+0000 7f5e40ea9ec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T08:01:54.283 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:01:54 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:01:54.281+0000 7f5e40ea9ec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T08:01:54.551 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:01:54 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:01:54.408+0000 7f5e40ea9ec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T08:01:54.552 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:01:54 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:01:54.547+0000 7f5e40ea9ec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T08:01:55.234 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:55 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/3746144202' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-11T08:01:55.234 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:55 smithi122 ceph-mon[28992]: mgrmap e5: y(active, since 9s) 2024-09-11T08:01:55.234 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:01:54 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:01:54.811+0000 7f5e40ea9ec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T08:01:55.234 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:01:54 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:01:54.872+0000 7f5e40ea9ec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T08:01:55.234 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:01:55 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:01:55.031+0000 7f5e40ea9ec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T08:01:56.234 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:01:55 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:01:55.868+0000 7f5e40ea9ec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T08:01:56.234 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:01:55 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:01:55.932+0000 7f5e40ea9ec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T08:01:56.933 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:01:56 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:01:56.642+0000 7f5e40ea9ec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T08:01:56.933 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:01:56 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:01:56.752+0000 7f5e40ea9ec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T08:01:56.933 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:01:56 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:01:56.872+0000 7f5e40ea9ec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T08:01:56.933 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:01:56 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:01:56.931+0000 7f5e40ea9ec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T08:01:57.234 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:01:57 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:01:57.056+0000 7f5e40ea9ec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T08:01:57.234 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:01:57 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:01:57.122+0000 7f5e40ea9ec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T08:01:57.234 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:01:57 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:01:57.179+0000 7f5e40ea9ec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T08:01:57.578 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:01:57 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:01:57.312+0000 7f5e40ea9ec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T08:01:57.579 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:01:57 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:01:57.375+0000 7f5e40ea9ec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T08:01:57.579 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:01:57 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:01:57.439+0000 7f5e40ea9ec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T08:01:57.579 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:01:57 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:01:57.520+0000 7f5e40ea9ec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T08:01:57.579 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:01:57 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:01:57.577+0000 7f5e40ea9ec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T08:01:57.862 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:01:57 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:01:57.635+0000 7f5e40ea9ec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T08:01:58.413 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:58 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/1178642086' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-11T08:01:58.680 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:01:58 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:01:58.411+0000 7f5e40ea9ec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T08:01:58.918 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout { 2024-09-11T08:01:58.919 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-11T08:01:58.919 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-11T08:01:58.919 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-09-11T08:01:58.919 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-11T08:01:58.919 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout } 2024-09-11T08:01:58.920 INFO:teuthology.orchestra.run.smithi122.stdout:Waiting for the mgr to restart... 2024-09-11T08:01:58.920 INFO:teuthology.orchestra.run.smithi122.stdout:Waiting for mgr epoch 5... 2024-09-11T08:01:59.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:59 smithi122 ceph-mon[28992]: Active manager daemon y restarted 2024-09-11T08:01:59.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:59 smithi122 ceph-mon[28992]: Activating manager daemon y 2024-09-11T08:01:59.735 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:59 smithi122 ceph-mon[28992]: osdmap e2: 0 total, 0 up, 0 in 2024-09-11T08:01:59.735 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:59 smithi122 ceph-mon[28992]: mgrmap e6: y(active, starting, since 0.0032908s) 2024-09-11T08:01:59.735 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:59 smithi122 ceph-mon[28992]: from='mgr.14116 172.21.15.122:0/381012618' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:01:59.735 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:59 smithi122 ceph-mon[28992]: from='mgr.14116 172.21.15.122:0/381012618' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T08:01:59.735 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:59 smithi122 ceph-mon[28992]: from='mgr.14116 172.21.15.122:0/381012618' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T08:01:59.735 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:59 smithi122 ceph-mon[28992]: from='mgr.14116 172.21.15.122:0/381012618' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T08:01:59.735 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:59 smithi122 ceph-mon[28992]: from='mgr.14116 172.21.15.122:0/381012618' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T08:01:59.735 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:59 smithi122 ceph-mon[28992]: Manager daemon y is now available 2024-09-11T08:01:59.735 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:59 smithi122 ceph-mon[28992]: Found migration_current of "None". Setting to last migration. 2024-09-11T08:01:59.735 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:59 smithi122 ceph-mon[28992]: from='mgr.14116 172.21.15.122:0/381012618' entity='mgr.y' 2024-09-11T08:01:59.735 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:59 smithi122 ceph-mon[28992]: from='mgr.14116 172.21.15.122:0/381012618' entity='mgr.y' 2024-09-11T08:01:59.735 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:59 smithi122 ceph-mon[28992]: from='mgr.14116 172.21.15.122:0/381012618' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:01:59.736 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:59 smithi122 ceph-mon[28992]: from='mgr.14116 172.21.15.122:0/381012618' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:01:59.736 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:59 smithi122 ceph-mon[28992]: from='mgr.14116 172.21.15.122:0/381012618' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-11T08:01:59.736 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:01:59 smithi122 ceph-mon[28992]: from='mgr.14116 172.21.15.122:0/381012618' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-11T08:02:00.026 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout { 2024-09-11T08:02:00.026 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-11T08:02:00.027 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-11T08:02:00.027 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout } 2024-09-11T08:02:00.027 INFO:teuthology.orchestra.run.smithi122.stdout:mgr epoch 5 is available 2024-09-11T08:02:00.027 INFO:teuthology.orchestra.run.smithi122.stdout:Setting orchestrator backend to cephadm... 2024-09-11T08:02:00.147 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:01:59 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:01:59.786+0000 7f5df3297640 -1 log_channel(cephadm) log [ERR] : [11/Sep/2024:08:01:59] ENGINE Error in HTTPServer.serve 2024-09-11T08:02:00.148 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:01:59 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: Traceback (most recent call last): 2024-09-11T08:02:00.148 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:01:59 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T08:02:00.148 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:01:59 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: self._connections.run(self.expiration_interval) 2024-09-11T08:02:00.148 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:01:59 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T08:02:00.148 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:01:59 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: self._run(expiration_interval) 2024-09-11T08:02:00.148 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:01:59 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T08:02:00.148 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:01:59 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T08:02:00.148 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:01:59 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T08:02:00.148 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:01:59 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T08:02:00.148 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:01:59 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T08:02:00.148 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:01:59 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: s = self.context.wrap_socket( 2024-09-11T08:02:00.148 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:01:59 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T08:02:00.149 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:01:59 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: return self.sslsocket_class._create( 2024-09-11T08:02:00.149 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:01:59 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T08:02:00.149 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:01:59 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: self.do_handshake() 2024-09-11T08:02:00.149 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:01:59 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T08:02:00.149 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:01:59 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: self._sslobj.do_handshake() 2024-09-11T08:02:00.149 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:01:59 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T08:02:00.149 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:01:59 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:02:00.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:00 smithi122 ceph-mon[28992]: from='client.14126 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-11T08:02:00.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:00 smithi122 ceph-mon[28992]: mgrmap e7: y(active, since 1.00513s) 2024-09-11T08:02:00.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:00 smithi122 ceph-mon[28992]: from='client.14126 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-11T08:02:00.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:00 smithi122 ceph-mon[28992]: [11/Sep/2024:08:01:59] ENGINE Bus STARTING 2024-09-11T08:02:00.735 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:00 smithi122 ceph-mon[28992]: [11/Sep/2024:08:01:59] ENGINE Serving on https://172.21.15.122:7150 2024-09-11T08:02:00.735 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:00 smithi122 ceph-mon[28992]: [11/Sep/2024:08:01:59] ENGINE Bus STARTED 2024-09-11T08:02:00.735 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:00 smithi122 ceph-mon[28992]: [11/Sep/2024:08:01:59] ENGINE Error in HTTPServer.serve 2024-09-11T08:02:00.735 INFO:journalctl@ceph.mon.a.smithi122.stdout: Traceback (most recent call last): 2024-09-11T08:02:00.735 INFO:journalctl@ceph.mon.a.smithi122.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T08:02:00.735 INFO:journalctl@ceph.mon.a.smithi122.stdout: self._connections.run(self.expiration_interval) 2024-09-11T08:02:00.735 INFO:journalctl@ceph.mon.a.smithi122.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T08:02:00.735 INFO:journalctl@ceph.mon.a.smithi122.stdout: self._run(expiration_interval) 2024-09-11T08:02:00.735 INFO:journalctl@ceph.mon.a.smithi122.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T08:02:00.735 INFO:journalctl@ceph.mon.a.smithi122.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T08:02:00.735 INFO:journalctl@ceph.mon.a.smithi122.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T08:02:00.735 INFO:journalctl@ceph.mon.a.smithi122.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T08:02:00.735 INFO:journalctl@ceph.mon.a.smithi122.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T08:02:00.735 INFO:journalctl@ceph.mon.a.smithi122.stdout: s = self.context.wrap_socket( 2024-09-11T08:02:00.735 INFO:journalctl@ceph.mon.a.smithi122.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T08:02:00.735 INFO:journalctl@ceph.mon.a.smithi122.stdout: return self.sslsocket_class._create( 2024-09-11T08:02:00.736 INFO:journalctl@ceph.mon.a.smithi122.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T08:02:00.736 INFO:journalctl@ceph.mon.a.smithi122.stdout: self.do_handshake() 2024-09-11T08:02:00.736 INFO:journalctl@ceph.mon.a.smithi122.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T08:02:00.736 INFO:journalctl@ceph.mon.a.smithi122.stdout: self._sslobj.do_handshake() 2024-09-11T08:02:00.736 INFO:journalctl@ceph.mon.a.smithi122.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T08:02:00.736 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:00 smithi122 ceph-mon[28992]: from='mgr.14116 172.21.15.122:0/381012618' entity='mgr.y' 2024-09-11T08:02:00.736 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:00 smithi122 ceph-mon[28992]: from='mgr.14116 172.21.15.122:0/381012618' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:02:01.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:01 smithi122 ceph-mon[28992]: from='client.14128 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:02:01.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:01 smithi122 ceph-mon[28992]: from='mgr.14116 172.21.15.122:0/381012618' entity='mgr.y' 2024-09-11T08:02:01.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:01 smithi122 ceph-mon[28992]: from='mgr.14116 172.21.15.122:0/381012618' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:02:01.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:01 smithi122 ceph-mon[28992]: mgrmap e8: y(active, since 2s) 2024-09-11T08:02:02.331 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-11T08:02:02.331 INFO:teuthology.orchestra.run.smithi122.stdout:Generating ssh key... 2024-09-11T08:02:02.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:02 smithi122 ceph-mon[28992]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:02:03.286 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:02:02 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: Generating public/private rsa key pair. 2024-09-11T08:02:03.286 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:02:02 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: Your identification has been saved in /tmp/tmpopvnazbu/key 2024-09-11T08:02:03.286 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:02:02 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: Your public key has been saved in /tmp/tmpopvnazbu/key.pub 2024-09-11T08:02:03.286 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:02:02 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: The key fingerprint is: 2024-09-11T08:02:03.286 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:02:02 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: SHA256:Ch+qebpu2QxYqDhJh0njnBzPZGtQkwIotrX5iB+6+UY ceph-f1d75850-7013-11ef-bcea-c7b262605968 2024-09-11T08:02:03.286 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:02:02 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: The key's randomart image is: 2024-09-11T08:02:03.286 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:02:02 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: +---[RSA 3072]----+ 2024-09-11T08:02:03.286 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:02:02 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: |+ .o. | 2024-09-11T08:02:03.286 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:02:02 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: |oB =. | 2024-09-11T08:02:03.287 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:02:02 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: |B./ + | 2024-09-11T08:02:03.287 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:02:02 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: |.X.O | 2024-09-11T08:02:03.287 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:02:02 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: |+++ + . S | 2024-09-11T08:02:03.287 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:02:02 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: |*..E = o | 2024-09-11T08:02:03.287 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:02:02 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: | .+=o o | 2024-09-11T08:02:03.287 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:02:02 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: | .+=+ | 2024-09-11T08:02:03.287 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:02:02 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: | =X= | 2024-09-11T08:02:03.287 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:02:02 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: +----[SHA256]-----+ 2024-09-11T08:02:04.110 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:03 smithi122 ceph-mon[28992]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:02:04.110 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:03 smithi122 ceph-mon[28992]: Generating ssh key... 2024-09-11T08:02:04.110 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:03 smithi122 ceph-mon[28992]: from='mgr.14116 172.21.15.122:0/381012618' entity='mgr.y' 2024-09-11T08:02:04.110 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:03 smithi122 ceph-mon[28992]: from='mgr.14116 172.21.15.122:0/381012618' entity='mgr.y' 2024-09-11T08:02:04.703 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDewtfuzsAsglILsWgH4fdN0lKq2n2CF/u2yrs5EQ50VLJ8cuazGB7FKwLJGubfOflsYD4Yr97l4yKrJ0AjjJIDB3nAaasBO86W20jekUqCXhAvhoVOP6MaUqJWt/2dm+SHcvTNlj6YAOrAvBta8dYNGnmVvkL9e/tbQJJOyWBUHA42OJ6DWsbs+OcA0scLF6im+SUl6wdYDpHF/KjP9ilnQYHxXkfJUHkM1+OCo8ZwSMnssRw3SCuwAfFHuyFlaYfSFUothgZjoORPvVYNrTQpsRAu5UmsSXDuFDYl31Mzp3mRcErWAuVa8xYb3REEof2lNX5VHIekPkinxCY8eto2AlT+zoFEHV06r4WCmncP3ooCIpd6ZKT/uYADYPgWoTI4swGibgjvsW8t7r0WQZj3sJG7e7mlB7kw9IzvMtCNkV9+JDmZj1+S9WPSs08GnE5PgKHAPcXhVQ9+B3MczgD8IXqjxvqVuHpi5YmhXRR0f8X/HneqpsJF6ymg6l3U7ek= ceph-f1d75850-7013-11ef-bcea-c7b262605968 2024-09-11T08:02:04.704 INFO:teuthology.orchestra.run.smithi122.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-11T08:02:04.704 INFO:teuthology.orchestra.run.smithi122.stdout:Adding key to root@localhost authorized_keys... 2024-09-11T08:02:04.706 INFO:teuthology.orchestra.run.smithi122.stdout:Adding host smithi122... 2024-09-11T08:02:05.392 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:05 smithi122 ceph-mon[28992]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:02:06.319 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:06 smithi122 ceph-mon[28992]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi122", "addr": "172.21.15.122", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:02:07.431 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:07 smithi122 ceph-mon[28992]: Deploying cephadm binary to smithi122 2024-09-11T08:02:08.738 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout Added host 'smithi122' with addr '172.21.15.122' 2024-09-11T08:02:08.739 INFO:teuthology.orchestra.run.smithi122.stdout:Deploying unmanaged mon service... 2024-09-11T08:02:09.299 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:08 smithi122 ceph-mon[28992]: from='mgr.14116 172.21.15.122:0/381012618' entity='mgr.y' 2024-09-11T08:02:09.299 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:08 smithi122 ceph-mon[28992]: Added host smithi122 2024-09-11T08:02:09.299 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:08 smithi122 ceph-mon[28992]: from='mgr.14116 172.21.15.122:0/381012618' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:02:10.050 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-11T08:02:10.050 INFO:teuthology.orchestra.run.smithi122.stdout:Deploying unmanaged mgr service... 2024-09-11T08:02:10.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:10 smithi122 ceph-mon[28992]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:02:10.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:10 smithi122 ceph-mon[28992]: Saving service mon spec with placement count:5 2024-09-11T08:02:10.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:10 smithi122 ceph-mon[28992]: from='mgr.14116 172.21.15.122:0/381012618' entity='mgr.y' 2024-09-11T08:02:11.936 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-11T08:02:12.287 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:11 smithi122 ceph-mon[28992]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:02:12.287 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:11 smithi122 ceph-mon[28992]: Saving service mgr spec with placement count:2 2024-09-11T08:02:12.287 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:11 smithi122 ceph-mon[28992]: from='mgr.14116 172.21.15.122:0/381012618' entity='mgr.y' 2024-09-11T08:02:12.287 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:11 smithi122 ceph-mon[28992]: from='mgr.14116 172.21.15.122:0/381012618' entity='mgr.y' 2024-09-11T08:02:12.287 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:11 smithi122 ceph-mon[28992]: from='mgr.14116 172.21.15.122:0/381012618' entity='mgr.y' 2024-09-11T08:02:13.778 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:13 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/1732690493' entity='client.admin' 2024-09-11T08:02:13.778 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:13 smithi122 ceph-mon[28992]: from='mgr.14116 172.21.15.122:0/381012618' entity='mgr.y' 2024-09-11T08:02:14.887 INFO:teuthology.orchestra.run.smithi122.stdout:Enabling the dashboard module... 2024-09-11T08:02:15.271 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:15 smithi122 ceph-mon[28992]: from='mgr.14116 172.21.15.122:0/381012618' entity='mgr.y' 2024-09-11T08:02:15.271 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:15 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/1060924269' entity='client.admin' 2024-09-11T08:02:16.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:16 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/2849084681' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-11T08:02:16.484 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:02:16 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: ignoring --setuser ceph since I am not root 2024-09-11T08:02:16.484 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:02:16 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: ignoring --setgroup ceph since I am not root 2024-09-11T08:02:16.484 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:02:16 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:02:16.377+0000 7f841c78eec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T08:02:16.484 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:02:16 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:02:16.433+0000 7f841c78eec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T08:02:16.761 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:02:16 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:02:16.499+0000 7f841c78eec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T08:02:16.762 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:02:16 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:02:16.624+0000 7f841c78eec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T08:02:16.762 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:02:16 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:02:16.760+0000 7f841c78eec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T08:02:17.363 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:17 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/2849084681' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-11T08:02:17.363 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:17 smithi122 ceph-mon[28992]: mgrmap e9: y(active, since 17s) 2024-09-11T08:02:17.363 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:02:17 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:02:17.019+0000 7f841c78eec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T08:02:17.363 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:02:17 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:02:17.080+0000 7f841c78eec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T08:02:17.363 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:02:17 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:02:17.247+0000 7f841c78eec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T08:02:18.339 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:02:18 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:02:18.128+0000 7f841c78eec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T08:02:18.339 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:02:18 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:02:18.192+0000 7f841c78eec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T08:02:18.339 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:18 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/870560245' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-11T08:02:18.342 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout { 2024-09-11T08:02:18.342 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-11T08:02:18.342 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-11T08:02:18.342 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-09-11T08:02:18.342 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-11T08:02:18.342 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout } 2024-09-11T08:02:18.342 INFO:teuthology.orchestra.run.smithi122.stdout:Waiting for the mgr to restart... 2024-09-11T08:02:18.343 INFO:teuthology.orchestra.run.smithi122.stdout:Waiting for mgr epoch 9... 2024-09-11T08:02:19.217 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:02:18 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:02:18.923+0000 7f841c78eec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T08:02:19.217 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:02:19 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:02:19.035+0000 7f841c78eec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T08:02:19.217 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:02:19 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:02:19.156+0000 7f841c78eec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T08:02:19.217 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:02:19 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:02:19.215+0000 7f841c78eec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T08:02:19.484 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:02:19 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:02:19.341+0000 7f841c78eec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T08:02:19.484 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:02:19 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:02:19.407+0000 7f841c78eec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T08:02:19.484 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:02:19 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:02:19.464+0000 7f841c78eec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T08:02:19.862 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:02:19 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:02:19.596+0000 7f841c78eec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T08:02:19.862 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:02:19 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:02:19.659+0000 7f841c78eec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T08:02:19.862 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:02:19 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:02:19.723+0000 7f841c78eec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T08:02:19.862 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:02:19 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:02:19.803+0000 7f841c78eec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T08:02:19.862 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:02:19 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:02:19.860+0000 7f841c78eec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T08:02:20.234 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:02:19 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:02:19.917+0000 7f841c78eec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T08:02:20.984 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:02:20 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:02:20.653+0000 7f841c78eec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T08:02:20.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:20 smithi122 ceph-mon[28992]: Active manager daemon y restarted 2024-09-11T08:02:20.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:20 smithi122 ceph-mon[28992]: Activating manager daemon y 2024-09-11T08:02:20.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:20 smithi122 ceph-mon[28992]: osdmap e3: 0 total, 0 up, 0 in 2024-09-11T08:02:20.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:20 smithi122 ceph-mon[28992]: mgrmap e10: y(active, starting, since 0.00324933s) 2024-09-11T08:02:20.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:20 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:02:20.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:20 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T08:02:20.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:20 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T08:02:20.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:20 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T08:02:20.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:20 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T08:02:20.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:20 smithi122 ceph-mon[28992]: Manager daemon y is now available 2024-09-11T08:02:20.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:20 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:02:20.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:20 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-11T08:02:20.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:20 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-11T08:02:21.985 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:02:21 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:02:21.784+0000 7f83cd2eb640 -1 log_channel(cephadm) log [ERR] : [11/Sep/2024:08:02:21] ENGINE Error in HTTPServer.serve 2024-09-11T08:02:21.985 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:02:21 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: Traceback (most recent call last): 2024-09-11T08:02:21.985 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:02:21 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T08:02:21.985 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:02:21 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: self._connections.run(self.expiration_interval) 2024-09-11T08:02:21.985 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:02:21 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T08:02:21.985 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:02:21 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: self._run(expiration_interval) 2024-09-11T08:02:21.985 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:02:21 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T08:02:21.985 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:02:21 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T08:02:21.985 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:02:21 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T08:02:21.986 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:02:21 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T08:02:21.986 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:02:21 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T08:02:21.986 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:02:21 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: s = self.context.wrap_socket( 2024-09-11T08:02:21.986 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:02:21 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T08:02:21.986 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:02:21 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: return self.sslsocket_class._create( 2024-09-11T08:02:21.986 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:02:21 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T08:02:21.986 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:02:21 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: self.do_handshake() 2024-09-11T08:02:21.986 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:02:21 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T08:02:21.986 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:02:21 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: self._sslobj.do_handshake() 2024-09-11T08:02:21.986 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:02:21 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T08:02:21.986 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:02:21 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:02:22.319 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout { 2024-09-11T08:02:22.320 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-11T08:02:22.320 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-11T08:02:22.320 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout } 2024-09-11T08:02:22.320 INFO:teuthology.orchestra.run.smithi122.stdout:mgr epoch 9 is available 2024-09-11T08:02:22.320 INFO:teuthology.orchestra.run.smithi122.stdout:Generating a dashboard self-signed certificate... 2024-09-11T08:02:22.703 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:22 smithi122 ceph-mon[28992]: mgrmap e11: y(active, since 1.00559s) 2024-09-11T08:02:22.704 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:22 smithi122 ceph-mon[28992]: [11/Sep/2024:08:02:21] ENGINE Bus STARTING 2024-09-11T08:02:22.704 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:22 smithi122 ceph-mon[28992]: [11/Sep/2024:08:02:21] ENGINE Serving on https://172.21.15.122:7150 2024-09-11T08:02:22.704 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:22 smithi122 ceph-mon[28992]: [11/Sep/2024:08:02:21] ENGINE Bus STARTED 2024-09-11T08:02:22.704 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:22 smithi122 ceph-mon[28992]: [11/Sep/2024:08:02:21] ENGINE Error in HTTPServer.serve 2024-09-11T08:02:22.704 INFO:journalctl@ceph.mon.a.smithi122.stdout: Traceback (most recent call last): 2024-09-11T08:02:22.704 INFO:journalctl@ceph.mon.a.smithi122.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T08:02:22.704 INFO:journalctl@ceph.mon.a.smithi122.stdout: self._connections.run(self.expiration_interval) 2024-09-11T08:02:22.704 INFO:journalctl@ceph.mon.a.smithi122.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T08:02:22.704 INFO:journalctl@ceph.mon.a.smithi122.stdout: self._run(expiration_interval) 2024-09-11T08:02:22.704 INFO:journalctl@ceph.mon.a.smithi122.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T08:02:22.705 INFO:journalctl@ceph.mon.a.smithi122.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T08:02:22.705 INFO:journalctl@ceph.mon.a.smithi122.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T08:02:22.705 INFO:journalctl@ceph.mon.a.smithi122.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T08:02:22.705 INFO:journalctl@ceph.mon.a.smithi122.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T08:02:22.705 INFO:journalctl@ceph.mon.a.smithi122.stdout: s = self.context.wrap_socket( 2024-09-11T08:02:22.705 INFO:journalctl@ceph.mon.a.smithi122.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T08:02:22.705 INFO:journalctl@ceph.mon.a.smithi122.stdout: return self.sslsocket_class._create( 2024-09-11T08:02:22.705 INFO:journalctl@ceph.mon.a.smithi122.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T08:02:22.705 INFO:journalctl@ceph.mon.a.smithi122.stdout: self.do_handshake() 2024-09-11T08:02:22.705 INFO:journalctl@ceph.mon.a.smithi122.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T08:02:22.705 INFO:journalctl@ceph.mon.a.smithi122.stdout: self._sslobj.do_handshake() 2024-09-11T08:02:22.705 INFO:journalctl@ceph.mon.a.smithi122.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T08:02:22.705 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:22 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:02:22.705 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:22 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:02:23.627 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-11T08:02:23.627 INFO:teuthology.orchestra.run.smithi122.stdout:Creating initial admin user... 2024-09-11T08:02:24.237 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:24 smithi122 ceph-mon[28992]: from='client.14160 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:02:24.237 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:24 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:02:24.237 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:24 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:02:24.237 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:24 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:02:24.237 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:24 smithi122 ceph-mon[28992]: mgrmap e12: y(active, since 2s) 2024-09-11T08:02:25.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:25 smithi122 ceph-mon[28992]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:02:25.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:25 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:02:25.541 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$pNMxl4LLDwp0IXcwLuMv2ebcExZDmyreHbQbsMvggOXJhVt4Il9iy", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726041744, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-11T08:02:25.541 INFO:teuthology.orchestra.run.smithi122.stdout:Fetching dashboard port number... 2024-09-11T08:02:26.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:26 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/1220280798' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-11T08:02:26.664 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout 8443 2024-09-11T08:02:26.675 INFO:teuthology.orchestra.run.smithi122.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-11T08:02:26.675 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stdout disabled 2024-09-11T08:02:26.682 INFO:teuthology.orchestra.run.smithi122.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-11T08:02:26.682 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stdout inactive 2024-09-11T08:02:26.683 INFO:teuthology.orchestra.run.smithi122.stdout:firewalld.service is not enabled 2024-09-11T08:02:26.683 INFO:teuthology.orchestra.run.smithi122.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-11T08:02:26.685 INFO:teuthology.orchestra.run.smithi122.stdout:Ceph Dashboard is now available at: 2024-09-11T08:02:26.685 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T08:02:26.685 INFO:teuthology.orchestra.run.smithi122.stdout: URL: https://smithi122.front.sepia.ceph.com:8443/ 2024-09-11T08:02:26.685 INFO:teuthology.orchestra.run.smithi122.stdout: User: admin 2024-09-11T08:02:26.685 INFO:teuthology.orchestra.run.smithi122.stdout: Password: 02vpnr6ieh 2024-09-11T08:02:26.685 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T08:02:26.685 INFO:teuthology.orchestra.run.smithi122.stdout:Saving cluster configuration to /var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/config directory 2024-09-11T08:02:26.686 INFO:teuthology.orchestra.run.smithi122.stdout:Enabling autotune for osd_memory_target 2024-09-11T08:02:28.279 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:28 smithi122 ceph-mon[28992]: mgrmap e13: y(active, since 6s) 2024-09-11T08:02:29.103 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-11T08:02:29.104 INFO:teuthology.orchestra.run.smithi122.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-11T08:02:29.104 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T08:02:29.104 INFO:teuthology.orchestra.run.smithi122.stdout: sudo /sbin/cephadm shell --fsid f1d75850-7013-11ef-bcea-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-11T08:02:29.104 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T08:02:29.104 INFO:teuthology.orchestra.run.smithi122.stdout:Or, if you are only running a single cluster on this host: 2024-09-11T08:02:29.104 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T08:02:29.104 INFO:teuthology.orchestra.run.smithi122.stdout: sudo /sbin/cephadm shell 2024-09-11T08:02:29.104 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T08:02:29.104 INFO:teuthology.orchestra.run.smithi122.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-11T08:02:29.104 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T08:02:29.104 INFO:teuthology.orchestra.run.smithi122.stdout: ceph telemetry on 2024-09-11T08:02:29.104 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T08:02:29.105 INFO:teuthology.orchestra.run.smithi122.stdout:For more information see: 2024-09-11T08:02:29.105 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T08:02:29.105 INFO:teuthology.orchestra.run.smithi122.stdout: https://docs.ceph.com/docs/master/mgr/telemetry/ 2024-09-11T08:02:29.105 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T08:02:29.105 INFO:teuthology.orchestra.run.smithi122.stdout:Bootstrap complete. 2024-09-11T08:02:29.140 INFO:tasks.cephadm:Fetching config... 2024-09-11T08:02:29.140 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-11T08:02:29.140 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-11T08:02:29.156 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-11T08:02:29.156 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-11T08:02:29.156 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-11T08:02:29.212 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-11T08:02:29.212 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-11T08:02:29.212 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-11T08:02:29.277 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-11T08:02:29.277 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-11T08:02:29.277 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-11T08:02:29.333 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-11T08:02:29.333 DEBUG:teuthology.orchestra.run.smithi122:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDewtfuzsAsglILsWgH4fdN0lKq2n2CF/u2yrs5EQ50VLJ8cuazGB7FKwLJGubfOflsYD4Yr97l4yKrJ0AjjJIDB3nAaasBO86W20jekUqCXhAvhoVOP6MaUqJWt/2dm+SHcvTNlj6YAOrAvBta8dYNGnmVvkL9e/tbQJJOyWBUHA42OJ6DWsbs+OcA0scLF6im+SUl6wdYDpHF/KjP9ilnQYHxXkfJUHkM1+OCo8ZwSMnssRw3SCuwAfFHuyFlaYfSFUothgZjoORPvVYNrTQpsRAu5UmsSXDuFDYl31Mzp3mRcErWAuVa8xYb3REEof2lNX5VHIekPkinxCY8eto2AlT+zoFEHV06r4WCmncP3ooCIpd6ZKT/uYADYPgWoTI4swGibgjvsW8t7r0WQZj3sJG7e7mlB7kw9IzvMtCNkV9+JDmZj1+S9WPSs08GnE5PgKHAPcXhVQ9+B3MczgD8IXqjxvqVuHpi5YmhXRR0f8X/HneqpsJF6ymg6l3U7ek= ceph-f1d75850-7013-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-11T08:02:29.408 INFO:teuthology.orchestra.run.smithi122.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDewtfuzsAsglILsWgH4fdN0lKq2n2CF/u2yrs5EQ50VLJ8cuazGB7FKwLJGubfOflsYD4Yr97l4yKrJ0AjjJIDB3nAaasBO86W20jekUqCXhAvhoVOP6MaUqJWt/2dm+SHcvTNlj6YAOrAvBta8dYNGnmVvkL9e/tbQJJOyWBUHA42OJ6DWsbs+OcA0scLF6im+SUl6wdYDpHF/KjP9ilnQYHxXkfJUHkM1+OCo8ZwSMnssRw3SCuwAfFHuyFlaYfSFUothgZjoORPvVYNrTQpsRAu5UmsSXDuFDYl31Mzp3mRcErWAuVa8xYb3REEof2lNX5VHIekPkinxCY8eto2AlT+zoFEHV06r4WCmncP3ooCIpd6ZKT/uYADYPgWoTI4swGibgjvsW8t7r0WQZj3sJG7e7mlB7kw9IzvMtCNkV9+JDmZj1+S9WPSs08GnE5PgKHAPcXhVQ9+B3MczgD8IXqjxvqVuHpi5YmhXRR0f8X/HneqpsJF6ymg6l3U7ek= ceph-f1d75850-7013-11ef-bcea-c7b262605968 2024-09-11T08:02:29.430 DEBUG:teuthology.orchestra.run.smithi135:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDewtfuzsAsglILsWgH4fdN0lKq2n2CF/u2yrs5EQ50VLJ8cuazGB7FKwLJGubfOflsYD4Yr97l4yKrJ0AjjJIDB3nAaasBO86W20jekUqCXhAvhoVOP6MaUqJWt/2dm+SHcvTNlj6YAOrAvBta8dYNGnmVvkL9e/tbQJJOyWBUHA42OJ6DWsbs+OcA0scLF6im+SUl6wdYDpHF/KjP9ilnQYHxXkfJUHkM1+OCo8ZwSMnssRw3SCuwAfFHuyFlaYfSFUothgZjoORPvVYNrTQpsRAu5UmsSXDuFDYl31Mzp3mRcErWAuVa8xYb3REEof2lNX5VHIekPkinxCY8eto2AlT+zoFEHV06r4WCmncP3ooCIpd6ZKT/uYADYPgWoTI4swGibgjvsW8t7r0WQZj3sJG7e7mlB7kw9IzvMtCNkV9+JDmZj1+S9WPSs08GnE5PgKHAPcXhVQ9+B3MczgD8IXqjxvqVuHpi5YmhXRR0f8X/HneqpsJF6ymg6l3U7ek= ceph-f1d75850-7013-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-11T08:02:29.565 INFO:teuthology.orchestra.run.smithi135.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDewtfuzsAsglILsWgH4fdN0lKq2n2CF/u2yrs5EQ50VLJ8cuazGB7FKwLJGubfOflsYD4Yr97l4yKrJ0AjjJIDB3nAaasBO86W20jekUqCXhAvhoVOP6MaUqJWt/2dm+SHcvTNlj6YAOrAvBta8dYNGnmVvkL9e/tbQJJOyWBUHA42OJ6DWsbs+OcA0scLF6im+SUl6wdYDpHF/KjP9ilnQYHxXkfJUHkM1+OCo8ZwSMnssRw3SCuwAfFHuyFlaYfSFUothgZjoORPvVYNrTQpsRAu5UmsSXDuFDYl31Mzp3mRcErWAuVa8xYb3REEof2lNX5VHIekPkinxCY8eto2AlT+zoFEHV06r4WCmncP3ooCIpd6ZKT/uYADYPgWoTI4swGibgjvsW8t7r0WQZj3sJG7e7mlB7kw9IzvMtCNkV9+JDmZj1+S9WPSs08GnE5PgKHAPcXhVQ9+B3MczgD8IXqjxvqVuHpi5YmhXRR0f8X/HneqpsJF6ymg6l3U7ek= ceph-f1d75850-7013-11ef-bcea-c7b262605968 2024-09-11T08:02:29.578 DEBUG:teuthology.orchestra.run.smithi122:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f1d75850-7013-11ef-bcea-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-11T08:02:29.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:29 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/1088074809' entity='client.admin' 2024-09-11T08:02:29.826 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:02:32.910 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-11T08:02:32.910 DEBUG:teuthology.orchestra.run.smithi122:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f1d75850-7013-11ef-bcea-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-11T08:02:33.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:33 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/1224436772' entity='client.admin' 2024-09-11T08:02:33.581 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:02:34.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:34 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:02:34.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:34 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:02:34.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:34 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi122", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:02:34.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:34 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:02:34.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:34 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:02:34.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:34 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:02:35.924 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi135 2024-09-11T08:02:35.924 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-09-11T08:02:35.924 DEBUG:teuthology.orchestra.run.smithi135:> dd of=/etc/ceph/ceph.conf 2024-09-11T08:02:35.945 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-09-11T08:02:35.945 DEBUG:teuthology.orchestra.run.smithi135:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-11T08:02:36.002 INFO:tasks.cephadm:Adding host smithi135 to orchestrator... 2024-09-11T08:02:36.003 DEBUG:teuthology.orchestra.run.smithi122:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f1d75850-7013-11ef-bcea-c7b262605968 -- ceph orch host add smithi135 2024-09-11T08:02:36.238 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:02:36.483 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:36 smithi122 ceph-mon[28992]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:02:36.483 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:36 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:02:36.483 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:36 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:02:36.483 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:36 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:02:36.483 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:36 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:02:36.483 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:36 smithi122 ceph-mon[28992]: Updating smithi122:/etc/ceph/ceph.conf 2024-09-11T08:02:36.483 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:36 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:02:36.483 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:36 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:02:36.483 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:36 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:02:37.465 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:37 smithi122 ceph-mon[28992]: Updating smithi122:/var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:02:37.465 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:37 smithi122 ceph-mon[28992]: Updating smithi122:/etc/ceph/ceph.client.admin.keyring 2024-09-11T08:02:37.465 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:37 smithi122 ceph-mon[28992]: Updating smithi122:/var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T08:02:38.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:38 smithi122 ceph-mon[28992]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi135", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:02:39.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:39 smithi122 ceph-mon[28992]: Deploying cephadm binary to smithi135 2024-09-11T08:02:40.676 INFO:teuthology.orchestra.run.smithi122.stdout:Added host 'smithi135' with addr '172.21.15.135' 2024-09-11T08:02:41.270 DEBUG:teuthology.orchestra.run.smithi122:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f1d75850-7013-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-11T08:02:41.504 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:02:41.958 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:41 smithi122 ceph-mon[28992]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:02:41.958 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:41 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:02:41.958 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:41 smithi122 ceph-mon[28992]: Added host smithi135 2024-09-11T08:02:41.958 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:41 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:02:41.958 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:41 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:02:41.958 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:41 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:02:43.217 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T08:02:43.218 INFO:teuthology.orchestra.run.smithi122.stdout:[{"addr": "172.21.15.122", "hostname": "smithi122", "labels": [], "status": ""}, {"addr": "172.21.15.135", "hostname": "smithi135", "labels": [], "status": ""}] 2024-09-11T08:02:43.218 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:43 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:02:43.818 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-11T08:02:43.818 DEBUG:teuthology.orchestra.run.smithi122:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f1d75850-7013-11ef-bcea-c7b262605968 -- ceph osd crush tunables default 2024-09-11T08:02:44.056 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:02:44.176 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:44 smithi122 ceph-mon[28992]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:02:44.176 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:44 smithi122 ceph-mon[28992]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:02:46.030 INFO:teuthology.orchestra.run.smithi122.stderr:adjusted tunables profile to default 2024-09-11T08:02:46.234 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:45 smithi122 ceph-mon[28992]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:02:46.234 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:45 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:02:46.234 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:45 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:02:46.234 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:45 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:02:46.234 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:45 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:02:46.234 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:45 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi135", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:02:46.234 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:45 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:02:46.234 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:45 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:02:46.235 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:45 smithi122 ceph-mon[28992]: Updating smithi135:/etc/ceph/ceph.conf 2024-09-11T08:02:46.235 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:45 smithi122 ceph-mon[28992]: Updating smithi135:/var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:02:46.235 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:45 smithi122 ceph-mon[28992]: Updating smithi135:/etc/ceph/ceph.client.admin.keyring 2024-09-11T08:02:46.235 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:45 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/3727770321' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-11T08:02:46.235 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:45 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:02:46.235 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:45 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:02:46.235 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:45 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:02:46.626 INFO:tasks.cephadm:Adding mon.a on smithi122 2024-09-11T08:02:46.626 INFO:tasks.cephadm:Adding mon.c on smithi122 2024-09-11T08:02:46.626 INFO:tasks.cephadm:Adding mon.b on smithi135 2024-09-11T08:02:46.626 DEBUG:teuthology.orchestra.run.smithi135:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f1d75850-7013-11ef-bcea-c7b262605968 -- ceph orch apply mon '3;smithi122:172.21.15.122=a;smithi122:[v2:172.21.15.122:3301,v1:172.21.15.122:6790]=c;smithi135:172.21.15.135=b' 2024-09-11T08:02:46.863 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:02:47.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:47 smithi122 ceph-mon[28992]: Updating smithi135:/var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T08:02:47.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:47 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/3727770321' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-11T08:02:47.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:47 smithi122 ceph-mon[28992]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T08:02:48.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:48 smithi122 ceph-mon[28992]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:02:48.488 INFO:teuthology.orchestra.run.smithi135.stdout:Scheduled mon update... 2024-09-11T08:02:49.322 DEBUG:teuthology.orchestra.run.smithi122:mon.c> sudo journalctl -f -n 0 -u ceph-f1d75850-7013-11ef-bcea-c7b262605968@mon.c.service 2024-09-11T08:02:49.325 DEBUG:teuthology.orchestra.run.smithi135:mon.b> sudo journalctl -f -n 0 -u ceph-f1d75850-7013-11ef-bcea-c7b262605968@mon.b.service 2024-09-11T08:02:49.328 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-11T08:02:49.328 DEBUG:teuthology.orchestra.run.smithi135:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f1d75850-7013-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-11T08:02:49.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:49 smithi122 ceph-mon[28992]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi122:172.21.15.122=a;smithi122:[v2:172.21.15.122:3301,v1:172.21.15.122:6790]=c;smithi135:172.21.15.135=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:02:49.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:49 smithi122 ceph-mon[28992]: Saving service mon spec with placement smithi122:172.21.15.122=a;smithi122:[v2:172.21.15.122:3301,v1:172.21.15.122:6790]=c;smithi135:172.21.15.135=b;count:3 2024-09-11T08:02:49.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:49 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:02:49.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:49 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:02:49.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:49 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:02:49.735 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:49 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:02:49.735 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:49 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:02:49.735 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:49 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T08:02:49.735 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:49 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:02:49.735 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:49 smithi122 ceph-mon[28992]: Deploying daemon mon.b on smithi135 2024-09-11T08:02:49.735 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:49 smithi122 ceph-mon[28992]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:02:50.374 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:02:51.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:51 smithi122 ceph-mon[28992]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:02:52.656 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:52 smithi135 systemd[1]: Starting Ceph mon.b for f1d75850-7013-11ef-bcea-c7b262605968... 2024-09-11T08:02:53.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 podman[32637]: 2024-09-11 08:02:52.946406304 +0000 UTC m=+0.016695560 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T08:02:53.352 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-11T08:02:53.352 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":1,"fsid":"f1d75850-7013-11ef-bcea-c7b262605968","modified":"2024-09-11T08:01:30.085674Z","created":"2024-09-11T08:01:30.085674Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:3300","nonce":0},{"type":"v1","addr":"172.21.15.122:6789","nonce":0}]},"addr":"172.21.15.122:6789/0","public_addr":"172.21.15.122:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-11T08:02:53.353 INFO:teuthology.orchestra.run.smithi135.stderr:dumped monmap epoch 1 2024-09-11T08:02:53.510 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 podman[32637]: 2024-09-11 08:02:53.242179761 +0000 UTC m=+0.312469003 container create fc64e10c38b93c28a2d645cf5a0bf09de5bae9dde4aac50fc81c78df280445a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-mon-b, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-11T08:02:53.510 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 podman[32637]: 2024-09-11 08:02:53.461438173 +0000 UTC m=+0.531727416 container init fc64e10c38b93c28a2d645cf5a0bf09de5bae9dde4aac50fc81c78df280445a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-mon-b, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-11T08:02:53.510 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 podman[32637]: 2024-09-11 08:02:53.464768004 +0000 UTC m=+0.535057249 container start fc64e10c38b93c28a2d645cf5a0bf09de5bae9dde4aac50fc81c78df280445a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-mon-b, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-11T08:02:53.510 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: set uid:gid to 167:167 (ceph:ceph) 2024-09-11T08:02:53.510 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: ceph version 17.2.7-1623-gd0e6828a (d0e6828a2016d48cf25ad84064e50742bb1c39b9) quincy (stable), process ceph-mon, pid 2 2024-09-11T08:02:53.510 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: pidfile_write: ignore empty --pid-file 2024-09-11T08:02:53.511 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: load: jerasure load: lrc 2024-09-11T08:02:53.511 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: RocksDB version: 6.15.5 2024-09-11T08:02:53.511 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-09-11T08:02:53.511 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Compile date Sep 10 2024 2024-09-11T08:02:53.511 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: DB SUMMARY 2024-09-11T08:02:53.511 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: DB Session ID: IQBDIL361L6JVJHMHMPU 2024-09-11T08:02:53.511 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: CURRENT file: CURRENT 2024-09-11T08:02:53.511 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: IDENTITY file: IDENTITY 2024-09-11T08:02:53.511 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: MANIFEST file: MANIFEST-000003 size: 57 Bytes 2024-09-11T08:02:53.511 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-09-11T08:02:53.511 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 511 ; 2024-09-11T08:02:53.511 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.error_if_exists: 0 2024-09-11T08:02:53.511 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.create_if_missing: 0 2024-09-11T08:02:53.513 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.paranoid_checks: 1 2024-09-11T08:02:53.513 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-11T08:02:53.513 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.env: 0x560e40a9c440 2024-09-11T08:02:53.513 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.fs: Posix File System 2024-09-11T08:02:53.513 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.info_log: 0x560e418d2ec0 2024-09-11T08:02:53.514 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.max_file_opening_threads: 16 2024-09-11T08:02:53.514 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.statistics: (nil) 2024-09-11T08:02:53.514 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.use_fsync: 0 2024-09-11T08:02:53.514 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.max_log_file_size: 0 2024-09-11T08:02:53.514 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-11T08:02:53.514 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-11T08:02:53.514 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.keep_log_file_num: 1000 2024-09-11T08:02:53.514 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.recycle_log_file_num: 0 2024-09-11T08:02:53.514 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.allow_fallocate: 1 2024-09-11T08:02:53.514 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.allow_mmap_reads: 0 2024-09-11T08:02:53.514 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.allow_mmap_writes: 0 2024-09-11T08:02:53.514 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.use_direct_reads: 0 2024-09-11T08:02:53.515 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-11T08:02:53.515 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.create_missing_column_families: 0 2024-09-11T08:02:53.515 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.db_log_dir: 2024-09-11T08:02:53.515 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-b/store.db 2024-09-11T08:02:53.515 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-11T08:02:53.515 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-11T08:02:53.515 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-11T08:02:53.515 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-11T08:02:53.515 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-11T08:02:53.515 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-11T08:02:53.515 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.advise_random_on_open: 1 2024-09-11T08:02:53.515 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.db_write_buffer_size: 0 2024-09-11T08:02:53.515 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.write_buffer_manager: 0x560e41990780 2024-09-11T08:02:53.516 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-11T08:02:53.516 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-09-11T08:02:53.516 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-11T08:02:53.516 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-11T08:02:53.516 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.rate_limiter: (nil) 2024-09-11T08:02:53.516 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-11T08:02:53.516 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.wal_recovery_mode: 2 2024-09-11T08:02:53.516 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.enable_thread_tracking: 0 2024-09-11T08:02:53.516 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.enable_pipelined_write: 0 2024-09-11T08:02:53.516 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.unordered_write: 0 2024-09-11T08:02:53.516 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-11T08:02:53.516 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-11T08:02:53.517 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-11T08:02:53.517 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-11T08:02:53.517 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.row_cache: None 2024-09-11T08:02:53.517 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.wal_filter: None 2024-09-11T08:02:53.517 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-11T08:02:53.517 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.allow_ingest_behind: 0 2024-09-11T08:02:53.517 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.preserve_deletes: 0 2024-09-11T08:02:53.517 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.two_write_queues: 0 2024-09-11T08:02:53.517 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.manual_wal_flush: 0 2024-09-11T08:02:53.517 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.atomic_flush: 0 2024-09-11T08:02:53.517 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-11T08:02:53.517 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-11T08:02:53.518 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-11T08:02:53.518 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.log_readahead_size: 0 2024-09-11T08:02:53.518 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-11T08:02:53.518 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.best_efforts_recovery: 0 2024-09-11T08:02:53.518 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-11T08:02:53.518 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-11T08:02:53.518 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.allow_data_in_errors: 0 2024-09-11T08:02:53.518 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.db_host_id: __hostname__ 2024-09-11T08:02:53.518 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.max_background_jobs: 2 2024-09-11T08:02:53.518 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.max_background_compactions: -1 2024-09-11T08:02:53.518 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.max_subcompactions: 1 2024-09-11T08:02:53.518 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-11T08:02:53.518 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-11T08:02:53.519 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-11T08:02:53.519 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.max_total_wal_size: 0 2024-09-11T08:02:53.519 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-11T08:02:53.519 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-11T08:02:53.519 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-11T08:02:53.519 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-11T08:02:53.519 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.max_open_files: -1 2024-09-11T08:02:53.519 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.bytes_per_sync: 0 2024-09-11T08:02:53.519 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-11T08:02:53.519 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-11T08:02:53.519 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.compaction_readahead_size: 0 2024-09-11T08:02:53.519 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.max_background_flushes: -1 2024-09-11T08:02:53.519 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Compression algorithms supported: 2024-09-11T08:02:53.520 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: kZSTD supported: 0 2024-09-11T08:02:53.520 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: kXpressCompression supported: 0 2024-09-11T08:02:53.520 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: kBZip2Compression supported: 0 2024-09-11T08:02:53.520 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-11T08:02:53.520 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: kLZ4Compression supported: 1 2024-09-11T08:02:53.520 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: kZlibCompression supported: 1 2024-09-11T08:02:53.520 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: kLZ4HCCompression supported: 1 2024-09-11T08:02:53.520 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: kSnappyCompression supported: 1 2024-09-11T08:02:53.520 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-11T08:02:53.520 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000003 2024-09-11T08:02:53.520 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2024-09-11T08:02:53.520 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-11T08:02:53.520 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.merge_operator: 2024-09-11T08:02:53.521 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.compaction_filter: None 2024-09-11T08:02:53.521 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.compaction_filter_factory: None 2024-09-11T08:02:53.521 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.sst_partitioner_factory: None 2024-09-11T08:02:53.521 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-11T08:02:53.521 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-11T08:02:53.521 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x560e41879a38) 2024-09-11T08:02:53.521 INFO:journalctl@ceph.mon.b.smithi135.stdout: cache_index_and_filter_blocks: 1 2024-09-11T08:02:53.521 INFO:journalctl@ceph.mon.b.smithi135.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-11T08:02:53.521 INFO:journalctl@ceph.mon.b.smithi135.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-11T08:02:53.521 INFO:journalctl@ceph.mon.b.smithi135.stdout: pin_top_level_index_and_filter: 1 2024-09-11T08:02:53.521 INFO:journalctl@ceph.mon.b.smithi135.stdout: index_type: 0 2024-09-11T08:02:53.521 INFO:journalctl@ceph.mon.b.smithi135.stdout: data_block_index_type: 0 2024-09-11T08:02:53.522 INFO:journalctl@ceph.mon.b.smithi135.stdout: index_shortening: 1 2024-09-11T08:02:53.522 INFO:journalctl@ceph.mon.b.smithi135.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-11T08:02:53.522 INFO:journalctl@ceph.mon.b.smithi135.stdout: hash_index_allow_collision: 1 2024-09-11T08:02:53.522 INFO:journalctl@ceph.mon.b.smithi135.stdout: checksum: 1 2024-09-11T08:02:53.522 INFO:journalctl@ceph.mon.b.smithi135.stdout: no_block_cache: 0 2024-09-11T08:02:53.522 INFO:journalctl@ceph.mon.b.smithi135.stdout: block_cache: 0x560e418af1f0 2024-09-11T08:02:53.522 INFO:journalctl@ceph.mon.b.smithi135.stdout: block_cache_name: BinnedLRUCache 2024-09-11T08:02:53.522 INFO:journalctl@ceph.mon.b.smithi135.stdout: block_cache_options: 2024-09-11T08:02:53.522 INFO:journalctl@ceph.mon.b.smithi135.stdout: capacity : 536870912 2024-09-11T08:02:53.522 INFO:journalctl@ceph.mon.b.smithi135.stdout: num_shard_bits : 4 2024-09-11T08:02:53.522 INFO:journalctl@ceph.mon.b.smithi135.stdout: strict_capacity_limit : 0 2024-09-11T08:02:53.523 INFO:journalctl@ceph.mon.b.smithi135.stdout: high_pri_pool_ratio: 0.000 2024-09-11T08:02:53.523 INFO:journalctl@ceph.mon.b.smithi135.stdout: block_cache_compressed: (nil) 2024-09-11T08:02:53.523 INFO:journalctl@ceph.mon.b.smithi135.stdout: persistent_cache: (nil) 2024-09-11T08:02:53.523 INFO:journalctl@ceph.mon.b.smithi135.stdout: block_size: 4096 2024-09-11T08:02:53.523 INFO:journalctl@ceph.mon.b.smithi135.stdout: block_size_deviation: 10 2024-09-11T08:02:53.523 INFO:journalctl@ceph.mon.b.smithi135.stdout: block_restart_interval: 16 2024-09-11T08:02:53.523 INFO:journalctl@ceph.mon.b.smithi135.stdout: index_block_restart_interval: 1 2024-09-11T08:02:53.523 INFO:journalctl@ceph.mon.b.smithi135.stdout: metadata_block_size: 4096 2024-09-11T08:02:53.523 INFO:journalctl@ceph.mon.b.smithi135.stdout: partition_filters: 0 2024-09-11T08:02:53.523 INFO:journalctl@ceph.mon.b.smithi135.stdout: use_delta_encoding: 1 2024-09-11T08:02:53.523 INFO:journalctl@ceph.mon.b.smithi135.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2024-09-11T08:02:53.523 INFO:journalctl@ceph.mon.b.smithi135.stdout: whole_key_filtering: 1 2024-09-11T08:02:53.523 INFO:journalctl@ceph.mon.b.smithi135.stdout: verify_compression: 0 2024-09-11T08:02:53.524 INFO:journalctl@ceph.mon.b.smithi135.stdout: read_amp_bytes_per_bit: 0 2024-09-11T08:02:53.524 INFO:journalctl@ceph.mon.b.smithi135.stdout: format_version: 4 2024-09-11T08:02:53.524 INFO:journalctl@ceph.mon.b.smithi135.stdout: enable_index_compression: 1 2024-09-11T08:02:53.524 INFO:journalctl@ceph.mon.b.smithi135.stdout: block_align: 0 2024-09-11T08:02:53.524 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.write_buffer_size: 33554432 2024-09-11T08:02:53.524 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.max_write_buffer_number: 2 2024-09-11T08:02:53.524 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.compression: NoCompression 2024-09-11T08:02:53.524 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.bottommost_compression: Disabled 2024-09-11T08:02:53.524 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.prefix_extractor: nullptr 2024-09-11T08:02:53.524 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-11T08:02:53.524 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.num_levels: 7 2024-09-11T08:02:53.524 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-11T08:02:53.524 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-11T08:02:53.525 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-11T08:02:53.525 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-11T08:02:53.525 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-11T08:02:53.525 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-11T08:02:53.525 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-11T08:02:53.525 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-11T08:02:53.525 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-11T08:02:53.525 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-11T08:02:53.525 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-11T08:02:53.525 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.compression_opts.level: 32767 2024-09-11T08:02:53.525 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.compression_opts.strategy: 0 2024-09-11T08:02:53.525 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-11T08:02:53.525 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-11T08:02:53.526 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-11T08:02:53.526 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.compression_opts.enabled: false 2024-09-11T08:02:53.526 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-11T08:02:53.526 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-11T08:02:53.526 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-11T08:02:53.526 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.target_file_size_base: 67108864 2024-09-11T08:02:53.526 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-11T08:02:53.526 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-11T08:02:53.526 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-11T08:02:53.526 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-11T08:02:53.527 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-11T08:02:53.527 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-11T08:02:53.527 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-11T08:02:53.527 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-11T08:02:53.527 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-11T08:02:53.527 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-11T08:02:53.527 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-11T08:02:53.527 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-11T08:02:53.527 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-11T08:02:53.527 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.arena_block_size: 4194304 2024-09-11T08:02:53.528 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-11T08:02:53.528 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-11T08:02:53.528 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-09-11T08:02:53.528 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.disable_auto_compactions: 0 2024-09-11T08:02:53.528 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-11T08:02:53.528 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-11T08:02:53.528 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-11T08:02:53.528 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-11T08:02:53.528 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-11T08:02:53.528 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-11T08:02:53.528 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-11T08:02:53.528 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-11T08:02:53.528 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-11T08:02:53.529 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-11T08:02:53.529 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-11T08:02:53.529 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.inplace_update_support: 0 2024-09-11T08:02:53.529 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-11T08:02:53.529 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-11T08:02:53.529 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-11T08:02:53.529 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-11T08:02:53.529 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.bloom_locality: 0 2024-09-11T08:02:53.529 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.max_successive_merges: 0 2024-09-11T08:02:53.529 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-11T08:02:53.529 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.paranoid_file_checks: 0 2024-09-11T08:02:53.530 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.force_consistency_checks: 1 2024-09-11T08:02:53.530 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.report_bg_io_stats: 0 2024-09-11T08:02:53.530 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.ttl: 2592000 2024-09-11T08:02:53.530 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-11T08:02:53.530 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.enable_blob_files: false 2024-09-11T08:02:53.530 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.min_blob_size: 0 2024-09-11T08:02:53.530 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.blob_file_size: 268435456 2024-09-11T08:02:53.530 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-11T08:02:53.530 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-11T08:02:53.530 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-11T08:02:53.531 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000003 succeeded,manifest_file_number is 3, next_file_number is 5, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-11T08:02:53.531 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 0 2024-09-11T08:02:53.531 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: [db/version_set.cc:4082] Creating manifest 7 2024-09-11T08:02:53.531 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726041773488908, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-11T08:02:53.531 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #4 mode 2 2024-09-11T08:02:53.531 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: [table/block_based/filter_policy.cc:991] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-09-11T08:02:53.531 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726041773489220, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1571, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1726041773, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "d2590399-6abd-4a82-ba6b-64cf8ad31930", "db_session_id": "IQBDIL361L6JVJHMHMPU"}} 2024-09-11T08:02:53.531 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: [db/version_set.cc:4082] Creating manifest 9 2024-09-11T08:02:53.531 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726041773489581, "job": 1, "event": "recovery_finished"} 2024-09-11T08:02:53.531 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-11T08:02:53.531 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x560e418e2700 2024-09-11T08:02:53.531 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: DB pointer 0x560e418b2000 2024-09-11T08:02:53.531 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2024-09-11T08:02:53.531 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: rocksdb: [db/db_impl/db_impl.cc:903] 2024-09-11T08:02:53.531 INFO:journalctl@ceph.mon.b.smithi135.stdout: ** DB Stats ** 2024-09-11T08:02:53.532 INFO:journalctl@ceph.mon.b.smithi135.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-11T08:02:53.532 INFO:journalctl@ceph.mon.b.smithi135.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-11T08:02:53.532 INFO:journalctl@ceph.mon.b.smithi135.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-11T08:02:53.532 INFO:journalctl@ceph.mon.b.smithi135.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-11T08:02:53.532 INFO:journalctl@ceph.mon.b.smithi135.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-11T08:02:53.532 INFO:journalctl@ceph.mon.b.smithi135.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-09-11T08:02:53.532 INFO:journalctl@ceph.mon.b.smithi135.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-11T08:02:53.532 INFO:journalctl@ceph.mon.b.smithi135.stdout: 2024-09-11T08:02:53.532 INFO:journalctl@ceph.mon.b.smithi135.stdout: ** Compaction Stats [default] ** 2024-09-11T08:02:53.532 INFO:journalctl@ceph.mon.b.smithi135.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-11T08:02:53.532 INFO:journalctl@ceph.mon.b.smithi135.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-11T08:02:53.532 INFO:journalctl@ceph.mon.b.smithi135.stdout: L0 1/0 1.53 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.0 0.00 0.00 1 0.000 0 0 2024-09-11T08:02:53.532 INFO:journalctl@ceph.mon.b.smithi135.stdout: Sum 1/0 1.53 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.0 0.00 0.00 1 0.000 0 0 2024-09-11T08:02:53.533 INFO:journalctl@ceph.mon.b.smithi135.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.0 0.00 0.00 1 0.000 0 0 2024-09-11T08:02:53.533 INFO:journalctl@ceph.mon.b.smithi135.stdout: 2024-09-11T08:02:53.533 INFO:journalctl@ceph.mon.b.smithi135.stdout: ** Compaction Stats [default] ** 2024-09-11T08:02:53.533 INFO:journalctl@ceph.mon.b.smithi135.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-11T08:02:53.533 INFO:journalctl@ceph.mon.b.smithi135.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-11T08:02:53.533 INFO:journalctl@ceph.mon.b.smithi135.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 5.0 0.00 0.00 1 0.000 0 0 2024-09-11T08:02:53.533 INFO:journalctl@ceph.mon.b.smithi135.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-11T08:02:53.533 INFO:journalctl@ceph.mon.b.smithi135.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-11T08:02:53.533 INFO:journalctl@ceph.mon.b.smithi135.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-11T08:02:53.533 INFO:journalctl@ceph.mon.b.smithi135.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-11T08:02:53.533 INFO:journalctl@ceph.mon.b.smithi135.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-11T08:02:53.533 INFO:journalctl@ceph.mon.b.smithi135.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-11T08:02:53.534 INFO:journalctl@ceph.mon.b.smithi135.stdout: Cumulative compaction: 0.00 GB write, 0.56 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T08:02:53.534 INFO:journalctl@ceph.mon.b.smithi135.stdout: Interval compaction: 0.00 GB write, 0.56 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T08:02:53.534 INFO:journalctl@ceph.mon.b.smithi135.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-11T08:02:53.534 INFO:journalctl@ceph.mon.b.smithi135.stdout: 2024-09-11T08:02:53.534 INFO:journalctl@ceph.mon.b.smithi135.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-11T08:02:53.534 INFO:journalctl@ceph.mon.b.smithi135.stdout: 2024-09-11T08:02:53.534 INFO:journalctl@ceph.mon.b.smithi135.stdout: ** Compaction Stats [default] ** 2024-09-11T08:02:53.534 INFO:journalctl@ceph.mon.b.smithi135.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-11T08:02:53.534 INFO:journalctl@ceph.mon.b.smithi135.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-11T08:02:53.534 INFO:journalctl@ceph.mon.b.smithi135.stdout: L0 1/0 1.53 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.0 0.00 0.00 1 0.000 0 0 2024-09-11T08:02:53.534 INFO:journalctl@ceph.mon.b.smithi135.stdout: Sum 1/0 1.53 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.0 0.00 0.00 1 0.000 0 0 2024-09-11T08:02:53.534 INFO:journalctl@ceph.mon.b.smithi135.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-09-11T08:02:53.534 INFO:journalctl@ceph.mon.b.smithi135.stdout: 2024-09-11T08:02:53.534 INFO:journalctl@ceph.mon.b.smithi135.stdout: ** Compaction Stats [default] ** 2024-09-11T08:02:53.535 INFO:journalctl@ceph.mon.b.smithi135.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-11T08:02:53.535 INFO:journalctl@ceph.mon.b.smithi135.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-11T08:02:53.535 INFO:journalctl@ceph.mon.b.smithi135.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 5.0 0.00 0.00 1 0.000 0 0 2024-09-11T08:02:53.535 INFO:journalctl@ceph.mon.b.smithi135.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-11T08:02:53.535 INFO:journalctl@ceph.mon.b.smithi135.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-11T08:02:53.535 INFO:journalctl@ceph.mon.b.smithi135.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-11T08:02:53.535 INFO:journalctl@ceph.mon.b.smithi135.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-11T08:02:53.535 INFO:journalctl@ceph.mon.b.smithi135.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-11T08:02:53.535 INFO:journalctl@ceph.mon.b.smithi135.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-11T08:02:53.535 INFO:journalctl@ceph.mon.b.smithi135.stdout: Cumulative compaction: 0.00 GB write, 0.55 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T08:02:53.535 INFO:journalctl@ceph.mon.b.smithi135.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T08:02:53.535 INFO:journalctl@ceph.mon.b.smithi135.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-11T08:02:53.535 INFO:journalctl@ceph.mon.b.smithi135.stdout: 2024-09-11T08:02:53.535 INFO:journalctl@ceph.mon.b.smithi135.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-11T08:02:53.535 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-09-11T08:02:53.536 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: using public_addr v2:172.21.15.135:0/0 -> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] 2024-09-11T08:02:53.536 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: starting mon.b rank -1 at public addrs [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] at bind addrs [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid f1d75850-7013-11ef-bcea-c7b262605968 2024-09-11T08:02:53.536 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: mon.b@-1(???) e0 preinit fsid f1d75850-7013-11ef-bcea-c7b262605968 2024-09-11T08:02:53.536 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: mon.b@-1(synchronizing).mds e1 new map 2024-09-11T08:02:53.536 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: mon.b@-1(synchronizing).mds e1 print_map 2024-09-11T08:02:53.536 INFO:journalctl@ceph.mon.b.smithi135.stdout: e1 2024-09-11T08:02:53.536 INFO:journalctl@ceph.mon.b.smithi135.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-11T08:02:53.536 INFO:journalctl@ceph.mon.b.smithi135.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-11T08:02:53.536 INFO:journalctl@ceph.mon.b.smithi135.stdout: legacy client fscid: -1 2024-09-11T08:02:53.536 INFO:journalctl@ceph.mon.b.smithi135.stdout: 2024-09-11T08:02:53.536 INFO:journalctl@ceph.mon.b.smithi135.stdout: No filesystems configured 2024-09-11T08:02:53.536 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-11T08:02:53.536 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-11T08:02:53.537 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-11T08:02:53.537 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-11T08:02:53.537 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-11T08:02:53.537 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-11T08:02:53.537 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-11T08:02:53.537 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T08:02:53.537 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T08:02:53.537 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T08:02:53.537 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi122:172.21.15.122=a;smithi122:[v2:172.21.15.122:3301,v1:172.21.15.122:6790]=c;smithi135:172.21.15.135=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:02:53.537 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: Saving service mon spec with placement smithi122:172.21.15.122=a;smithi122:[v2:172.21.15.122:3301,v1:172.21.15.122:6790]=c;smithi135:172.21.15.135=b;count:3 2024-09-11T08:02:53.537 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:02:53.537 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:02:53.537 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:02:53.538 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:02:53.538 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:02:53.538 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T08:02:53.538 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:02:53.538 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: Deploying daemon mon.b on smithi135 2024-09-11T08:02:53.538 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:02:53.538 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:02:53.538 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 ceph-mon[32689]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-11T08:02:53.925 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 bash[32637]: fc64e10c38b93c28a2d645cf5a0bf09de5bae9dde4aac50fc81c78df280445a7 2024-09-11T08:02:53.925 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:53 smithi135 systemd[1]: Started Ceph mon.b for f1d75850-7013-11ef-bcea-c7b262605968. 2024-09-11T08:02:55.175 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-11T08:02:55.176 DEBUG:teuthology.orchestra.run.smithi135:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f1d75850-7013-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-11T08:02:55.436 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:02:58.969 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:58 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:02:58.969 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:58 smithi122 ceph-mon[28992]: mon.a calling monitor election 2024-09-11T08:02:58.969 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:58 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:02:58.969 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:58 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:02:58.969 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:58 smithi122 ceph-mon[28992]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:02:58.969 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:58 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:02:58.970 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:58 smithi122 ceph-mon[28992]: mon.b calling monitor election 2024-09-11T08:02:58.970 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:58 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:02:58.970 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:58 smithi122 ceph-mon[28992]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:02:58.970 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:58 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:02:58.970 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:58 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:02:58.970 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:58 smithi122 ceph-mon[28992]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-11T08:02:58.970 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:58 smithi122 ceph-mon[28992]: monmap e2: 2 mons at {a=[v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0],b=[v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-11T08:02:58.970 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:58 smithi122 ceph-mon[28992]: fsmap 2024-09-11T08:02:58.970 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:58 smithi122 ceph-mon[28992]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T08:02:58.970 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:58 smithi122 ceph-mon[28992]: mgrmap e13: y(active, since 37s) 2024-09-11T08:02:58.970 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:58 smithi122 ceph-mon[28992]: overall HEALTH_OK 2024-09-11T08:02:58.970 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:58 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:02:58.970 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:58 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:02:58.971 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:58 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:02:58.971 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:58 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T08:02:58.971 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:58 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:02:58.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:58 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:02:58.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:58 smithi135 ceph-mon[32689]: mon.a calling monitor election 2024-09-11T08:02:58.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:58 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:02:58.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:58 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:02:58.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:58 smithi135 ceph-mon[32689]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:02:58.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:58 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:02:58.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:58 smithi135 ceph-mon[32689]: mon.b calling monitor election 2024-09-11T08:02:58.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:58 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:02:58.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:58 smithi135 ceph-mon[32689]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:02:58.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:58 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:02:58.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:58 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:02:58.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:58 smithi135 ceph-mon[32689]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-11T08:02:58.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:58 smithi135 ceph-mon[32689]: monmap e2: 2 mons at {a=[v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0],b=[v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-11T08:02:58.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:58 smithi135 ceph-mon[32689]: fsmap 2024-09-11T08:02:58.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:58 smithi135 ceph-mon[32689]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T08:02:58.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:58 smithi135 ceph-mon[32689]: mgrmap e13: y(active, since 37s) 2024-09-11T08:02:58.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:58 smithi135 ceph-mon[32689]: overall HEALTH_OK 2024-09-11T08:02:58.980 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:58 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:02:58.980 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:58 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:02:58.980 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:58 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:02:58.980 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:58 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T08:02:58.980 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:58 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:02:59.872 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:59 smithi122 ceph-mon[28992]: Deploying daemon mon.c on smithi122 2024-09-11T08:02:59.872 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:59 smithi122 ceph-mon[28992]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:02:59.872 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:02:59 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:02:59.882 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:59 smithi135 ceph-mon[32689]: Deploying daemon mon.c on smithi122 2024-09-11T08:02:59.882 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:59 smithi135 ceph-mon[32689]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:02:59.882 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:02:59 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:03:00.146 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-11T08:03:00.147 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":2,"fsid":"f1d75850-7013-11ef-bcea-c7b262605968","modified":"2024-09-11T08:02:53.514314Z","created":"2024-09-11T08:01:30.085674Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:3300","nonce":0},{"type":"v1","addr":"172.21.15.122:6789","nonce":0}]},"addr":"172.21.15.122:6789/0","public_addr":"172.21.15.122:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:3300","nonce":0},{"type":"v1","addr":"172.21.15.135:6789","nonce":0}]},"addr":"172.21.15.135:6789/0","public_addr":"172.21.15.135:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-11T08:03:00.147 INFO:teuthology.orchestra.run.smithi135.stderr:dumped monmap epoch 2 2024-09-11T08:03:00.702 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:00 smithi135 ceph-mon[32689]: from='client.? 172.21.15.135:0/2176968138' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T08:03:00.888 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:00 smithi122 ceph-mon[28992]: from='client.? 172.21.15.135:0/2176968138' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T08:03:01.727 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-11T08:03:01.728 DEBUG:teuthology.orchestra.run.smithi135:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f1d75850-7013-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-11T08:03:01.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:01 smithi135 ceph-mon[32689]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:03:01.981 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:03:01.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:01 smithi122 ceph-mon[28992]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:03:01.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:01 smithi122 systemd[1]: Started Ceph mon.c for f1d75850-7013-11ef-bcea-c7b262605968. 2024-09-11T08:03:07.229 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:06 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:03:07.229 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:06 smithi135 ceph-mon[32689]: mon.a calling monitor election 2024-09-11T08:03:07.229 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:06 smithi135 ceph-mon[32689]: mon.b calling monitor election 2024-09-11T08:03:07.230 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:06 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:03:07.230 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:06 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:03:07.230 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:06 smithi135 ceph-mon[32689]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:03:07.230 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:06 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:03:07.230 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:06 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:03:07.230 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:06 smithi135 ceph-mon[32689]: mon.c calling monitor election 2024-09-11T08:03:07.230 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:06 smithi135 ceph-mon[32689]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:03:07.231 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:06 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:03:07.231 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:06 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:03:07.231 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:06 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:03:07.231 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:06 smithi135 ceph-mon[32689]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-11T08:03:07.231 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:06 smithi135 ceph-mon[32689]: monmap e3: 3 mons at {a=[v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0],b=[v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0],c=[v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-11T08:03:07.231 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:06 smithi135 ceph-mon[32689]: fsmap 2024-09-11T08:03:07.232 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:06 smithi135 ceph-mon[32689]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T08:03:07.232 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:06 smithi135 ceph-mon[32689]: mgrmap e13: y(active, since 46s) 2024-09-11T08:03:07.232 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:06 smithi135 ceph-mon[32689]: overall HEALTH_OK 2024-09-11T08:03:07.232 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:06 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:07.232 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:06 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:07.232 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:06 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:07.232 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:06 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:07.234 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:06 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:03:07.235 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:06 smithi122 ceph-mon[28992]: mon.a calling monitor election 2024-09-11T08:03:07.235 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:06 smithi122 ceph-mon[28992]: mon.b calling monitor election 2024-09-11T08:03:07.235 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:06 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:03:07.235 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:06 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:03:07.235 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:06 smithi122 ceph-mon[28992]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:03:07.235 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:06 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:03:07.235 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:06 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:03:07.236 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:06 smithi122 ceph-mon[28992]: mon.c calling monitor election 2024-09-11T08:03:07.236 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:06 smithi122 ceph-mon[28992]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:03:07.236 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:06 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:03:07.236 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:06 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:03:07.236 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:06 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:03:07.236 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:06 smithi122 ceph-mon[28992]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-11T08:03:07.237 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:06 smithi122 ceph-mon[28992]: monmap e3: 3 mons at {a=[v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0],b=[v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0],c=[v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-11T08:03:07.237 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:06 smithi122 ceph-mon[28992]: fsmap 2024-09-11T08:03:07.237 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:06 smithi122 ceph-mon[28992]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T08:03:07.237 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:06 smithi122 ceph-mon[28992]: mgrmap e13: y(active, since 46s) 2024-09-11T08:03:07.237 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:06 smithi122 ceph-mon[28992]: overall HEALTH_OK 2024-09-11T08:03:07.238 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:06 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:07.238 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:06 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:07.238 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:06 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:07.238 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:06 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:07.238 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:06 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:03:07.239 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:06 smithi122 ceph-mon[39726]: mon.a calling monitor election 2024-09-11T08:03:07.239 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:06 smithi122 ceph-mon[39726]: mon.b calling monitor election 2024-09-11T08:03:07.239 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:06 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:03:07.239 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:06 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:03:07.239 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:06 smithi122 ceph-mon[39726]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:03:07.239 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:06 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:03:07.240 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:06 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:03:07.240 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:06 smithi122 ceph-mon[39726]: mon.c calling monitor election 2024-09-11T08:03:07.240 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:06 smithi122 ceph-mon[39726]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:03:07.240 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:06 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:03:07.240 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:06 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:03:07.240 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:06 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:03:07.241 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:06 smithi122 ceph-mon[39726]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-11T08:03:07.241 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:06 smithi122 ceph-mon[39726]: monmap e3: 3 mons at {a=[v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0],b=[v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0],c=[v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-11T08:03:07.241 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:06 smithi122 ceph-mon[39726]: fsmap 2024-09-11T08:03:07.241 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:06 smithi122 ceph-mon[39726]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T08:03:07.241 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:06 smithi122 ceph-mon[39726]: mgrmap e13: y(active, since 46s) 2024-09-11T08:03:07.241 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:06 smithi122 ceph-mon[39726]: overall HEALTH_OK 2024-09-11T08:03:07.242 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:06 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:07.242 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:06 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:07.242 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:06 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:07.242 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:06 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:07.285 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-11T08:03:07.285 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":3,"fsid":"f1d75850-7013-11ef-bcea-c7b262605968","modified":"2024-09-11T08:03:01.731405Z","created":"2024-09-11T08:01:30.085674Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:3300","nonce":0},{"type":"v1","addr":"172.21.15.122:6789","nonce":0}]},"addr":"172.21.15.122:6789/0","public_addr":"172.21.15.122:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:3300","nonce":0},{"type":"v1","addr":"172.21.15.135:6789","nonce":0}]},"addr":"172.21.15.135:6789/0","public_addr":"172.21.15.135:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:3301","nonce":0},{"type":"v1","addr":"172.21.15.122:6790","nonce":0}]},"addr":"172.21.15.122:6790/0","public_addr":"172.21.15.122:6790/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-09-11T08:03:07.285 INFO:teuthology.orchestra.run.smithi135.stderr:dumped monmap epoch 3 2024-09-11T08:03:08.157 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:07 smithi135 ceph-mon[32689]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:03:08.158 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:07 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:03:08.158 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:07 smithi135 ceph-mon[32689]: from='client.? 172.21.15.135:0/1883964910' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T08:03:08.158 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:07 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:03:08.234 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:07 smithi122 ceph-mon[28992]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:03:08.234 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:07 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:03:08.234 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:07 smithi122 ceph-mon[28992]: from='client.? 172.21.15.135:0/1883964910' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T08:03:08.234 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:07 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:03:08.234 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:07 smithi122 ceph-mon[39726]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:03:08.234 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:07 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:03:08.235 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:07 smithi122 ceph-mon[39726]: from='client.? 172.21.15.135:0/1883964910' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T08:03:08.235 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:07 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:03:08.516 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-11T08:03:08.516 DEBUG:teuthology.orchestra.run.smithi122:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f1d75850-7013-11ef-bcea-c7b262605968 -- ceph config generate-minimal-conf 2024-09-11T08:03:08.764 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:03:09.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:08 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:09.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:08 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:09.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:08 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:09.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:08 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:09.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:08 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:03:09.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:08 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:03:09.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:08 smithi122 ceph-mon[28992]: Updating smithi122:/etc/ceph/ceph.conf 2024-09-11T08:03:09.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:08 smithi122 ceph-mon[28992]: Updating smithi135:/etc/ceph/ceph.conf 2024-09-11T08:03:09.210 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:08 smithi122 ceph-mon[28992]: Updating smithi122:/var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:03:09.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:08 smithi122 ceph-mon[28992]: Updating smithi135:/var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:03:09.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:08 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:09.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:08 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:09.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:08 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:09.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:08 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:09.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:08 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:09.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:08 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T08:03:09.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:08 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T08:03:09.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:08 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:03:09.211 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:08 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:09.211 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:08 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:09.211 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:08 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:09.212 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:08 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:09.212 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:08 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:03:09.212 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:08 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:03:09.212 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:08 smithi122 ceph-mon[39726]: Updating smithi122:/etc/ceph/ceph.conf 2024-09-11T08:03:09.212 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:08 smithi122 ceph-mon[39726]: Updating smithi135:/etc/ceph/ceph.conf 2024-09-11T08:03:09.212 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:08 smithi122 ceph-mon[39726]: Updating smithi122:/var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:03:09.212 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:08 smithi122 ceph-mon[39726]: Updating smithi135:/var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:03:09.212 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:08 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:09.212 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:08 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:09.212 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:08 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:09.212 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:08 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:09.212 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:08 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:09.212 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:08 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T08:03:09.213 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:08 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T08:03:09.213 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:08 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:03:09.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:08 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:09.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:08 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:09.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:08 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:09.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:08 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:09.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:08 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:03:09.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:08 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:03:09.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:08 smithi135 ceph-mon[32689]: Updating smithi122:/etc/ceph/ceph.conf 2024-09-11T08:03:09.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:08 smithi135 ceph-mon[32689]: Updating smithi135:/etc/ceph/ceph.conf 2024-09-11T08:03:09.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:08 smithi135 ceph-mon[32689]: Updating smithi122:/var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:03:09.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:08 smithi135 ceph-mon[32689]: Updating smithi135:/var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:03:09.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:08 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:09.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:08 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:09.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:08 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:09.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:08 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:09.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:08 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:09.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:08 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T08:03:09.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:08 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T08:03:09.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:08 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:03:10.141 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:09 smithi122 ceph-mon[28992]: Reconfiguring mon.a (unknown last config time)... 2024-09-11T08:03:10.141 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:09 smithi122 ceph-mon[28992]: Reconfiguring daemon mon.a on smithi122 2024-09-11T08:03:10.141 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:09 smithi122 ceph-mon[28992]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:03:10.142 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:09 smithi122 ceph-mon[39726]: Reconfiguring mon.a (unknown last config time)... 2024-09-11T08:03:10.142 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:09 smithi122 ceph-mon[39726]: Reconfiguring daemon mon.a on smithi122 2024-09-11T08:03:10.142 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:09 smithi122 ceph-mon[39726]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:03:10.403 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:09 smithi135 ceph-mon[32689]: Reconfiguring mon.a (unknown last config time)... 2024-09-11T08:03:10.404 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:09 smithi135 ceph-mon[32689]: Reconfiguring daemon mon.a on smithi122 2024-09-11T08:03:10.404 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:09 smithi135 ceph-mon[32689]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:03:11.476 INFO:teuthology.orchestra.run.smithi122.stdout:# minimal ceph.conf for f1d75850-7013-11ef-bcea-c7b262605968 2024-09-11T08:03:11.476 INFO:teuthology.orchestra.run.smithi122.stdout:[global] 2024-09-11T08:03:11.476 INFO:teuthology.orchestra.run.smithi122.stdout: fsid = f1d75850-7013-11ef-bcea-c7b262605968 2024-09-11T08:03:11.476 INFO:teuthology.orchestra.run.smithi122.stdout: mon_host = [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] 2024-09-11T08:03:11.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:11 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:11.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:11 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:11.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:11 smithi135 ceph-mon[32689]: Reconfiguring mon.c (monmap changed)... 2024-09-11T08:03:11.729 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:11 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T08:03:11.729 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:11 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T08:03:11.729 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:11 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:03:11.729 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:11 smithi135 ceph-mon[32689]: Reconfiguring daemon mon.c on smithi122 2024-09-11T08:03:11.729 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:11 smithi135 ceph-mon[32689]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:03:11.729 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:11 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:11.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:11 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:11.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:11 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:11.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:11 smithi122 ceph-mon[28992]: Reconfiguring mon.c (monmap changed)... 2024-09-11T08:03:11.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:11 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T08:03:11.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:11 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T08:03:11.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:11 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:03:11.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:11 smithi122 ceph-mon[28992]: Reconfiguring daemon mon.c on smithi122 2024-09-11T08:03:11.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:11 smithi122 ceph-mon[28992]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:03:11.735 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:11 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:11.735 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:11 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:11.735 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:11 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:11.735 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:11 smithi122 ceph-mon[39726]: Reconfiguring mon.c (monmap changed)... 2024-09-11T08:03:11.735 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:11 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T08:03:11.735 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:11 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T08:03:11.735 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:11 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:03:11.735 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:11 smithi122 ceph-mon[39726]: Reconfiguring daemon mon.c on smithi122 2024-09-11T08:03:11.735 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:11 smithi122 ceph-mon[39726]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:03:11.735 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:11 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:12.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:12 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/77290861' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:03:12.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:12 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/77290861' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:03:12.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:12 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/77290861' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:03:12.900 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-11T08:03:12.901 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-11T08:03:12.901 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/etc/ceph/ceph.conf 2024-09-11T08:03:12.929 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-11T08:03:12.929 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-11T08:03:12.994 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-09-11T08:03:12.994 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/etc/ceph/ceph.conf 2024-09-11T08:03:13.041 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-09-11T08:03:13.041 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-11T08:03:13.107 INFO:tasks.cephadm:Adding mgr.y on smithi122 2024-09-11T08:03:13.108 INFO:tasks.cephadm:Adding mgr.x on smithi135 2024-09-11T08:03:13.108 DEBUG:teuthology.orchestra.run.smithi135:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f1d75850-7013-11ef-bcea-c7b262605968 -- ceph orch apply mgr '2;smithi122=y;smithi135=x' 2024-09-11T08:03:13.490 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:03:13.864 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:13 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:13.864 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:13 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:13.864 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:13 smithi135 ceph-mon[32689]: Reconfiguring mon.b (monmap changed)... 2024-09-11T08:03:13.865 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:13 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T08:03:13.865 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:13 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T08:03:13.865 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:13 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:03:13.865 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:13 smithi135 ceph-mon[32689]: Reconfiguring daemon mon.b on smithi135 2024-09-11T08:03:13.865 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:13 smithi135 ceph-mon[32689]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:03:13.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:13 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:13.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:13 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:13.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:13 smithi122 ceph-mon[28992]: Reconfiguring mon.b (monmap changed)... 2024-09-11T08:03:13.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:13 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T08:03:13.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:13 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T08:03:13.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:13 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:03:13.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:13 smithi122 ceph-mon[28992]: Reconfiguring daemon mon.b on smithi135 2024-09-11T08:03:13.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:13 smithi122 ceph-mon[28992]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:03:13.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:13 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:13.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:13 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:13.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:13 smithi122 ceph-mon[39726]: Reconfiguring mon.b (monmap changed)... 2024-09-11T08:03:13.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:13 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T08:03:13.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:13 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T08:03:13.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:13 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:03:13.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:13 smithi122 ceph-mon[39726]: Reconfiguring daemon mon.b on smithi135 2024-09-11T08:03:13.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:13 smithi122 ceph-mon[39726]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:03:15.169 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:14 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:15.169 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:14 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:15.169 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:14 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:03:15.169 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:14 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:03:15.169 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:14 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:03:15.169 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:14 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:15.234 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:14 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:15.234 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:14 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:15.234 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:14 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:03:15.234 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:14 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:03:15.235 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:14 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:03:15.235 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:14 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:15.235 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:14 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:15.235 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:14 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:15.235 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:14 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:03:15.235 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:14 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:03:15.235 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:14 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:03:15.235 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:14 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:15.529 INFO:teuthology.orchestra.run.smithi135.stdout:Scheduled mgr update... 2024-09-11T08:03:16.069 DEBUG:teuthology.orchestra.run.smithi135:mgr.x> sudo journalctl -f -n 0 -u ceph-f1d75850-7013-11ef-bcea-c7b262605968@mgr.x.service 2024-09-11T08:03:16.071 INFO:tasks.cephadm:Deploying OSDs... 2024-09-11T08:03:16.071 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-11T08:03:16.072 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/scratch_devs of=/dev/stdout 2024-09-11T08:03:16.091 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-11T08:03:16.091 DEBUG:teuthology.orchestra.run.smithi122:> stat /dev/vg_nvme/lv_1 2024-09-11T08:03:16.149 INFO:teuthology.orchestra.run.smithi122.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-11T08:03:16.150 INFO:teuthology.orchestra.run.smithi122.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T08:03:16.150 INFO:teuthology.orchestra.run.smithi122.stdout:Device: 5h/5d Inode: 972 Links: 1 2024-09-11T08:03:16.150 INFO:teuthology.orchestra.run.smithi122.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T08:03:16.150 INFO:teuthology.orchestra.run.smithi122.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T08:03:16.150 INFO:teuthology.orchestra.run.smithi122.stdout:Access: 2024-09-11 08:03:01.192468413 +0000 2024-09-11T08:03:16.150 INFO:teuthology.orchestra.run.smithi122.stdout:Modify: 2024-09-11 07:58:18.616876452 +0000 2024-09-11T08:03:16.150 INFO:teuthology.orchestra.run.smithi122.stdout:Change: 2024-09-11 07:58:18.616876452 +0000 2024-09-11T08:03:16.150 INFO:teuthology.orchestra.run.smithi122.stdout: Birth: 2024-09-11 07:58:18.616876452 +0000 2024-09-11T08:03:16.150 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-11T08:03:16.203 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:15 smithi135 ceph-mon[32689]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:03:16.203 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:15 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:16.203 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:15 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:03:16.203 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:15 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:03:16.203 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:15 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:03:16.203 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:15 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:16.203 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:15 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T08:03:16.203 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:15 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-11T08:03:16.203 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:15 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T08:03:16.203 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:15 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:03:16.216 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:15 smithi122 ceph-mon[28992]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:03:16.216 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:15 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:16.216 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:15 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:03:16.216 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:15 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:03:16.216 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:15 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:03:16.216 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:15 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:16.216 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:15 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T08:03:16.217 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:15 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-11T08:03:16.217 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:15 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T08:03:16.217 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:15 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:03:16.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:15 smithi122 ceph-mon[39726]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:03:16.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:15 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:16.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:15 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:03:16.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:15 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:03:16.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:15 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:03:16.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:15 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:16.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:15 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T08:03:16.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:15 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-11T08:03:16.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:15 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T08:03:16.218 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:15 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:03:16.219 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records in 2024-09-11T08:03:16.219 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records out 2024-09-11T08:03:16.219 INFO:teuthology.orchestra.run.smithi122.stderr:512 bytes copied, 0.000297133 s, 1.7 MB/s 2024-09-11T08:03:16.220 DEBUG:teuthology.orchestra.run.smithi122:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-11T08:03:16.280 DEBUG:teuthology.orchestra.run.smithi122:> stat /dev/vg_nvme/lv_2 2024-09-11T08:03:16.339 INFO:teuthology.orchestra.run.smithi122.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-11T08:03:16.339 INFO:teuthology.orchestra.run.smithi122.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T08:03:16.339 INFO:teuthology.orchestra.run.smithi122.stdout:Device: 5h/5d Inode: 984 Links: 1 2024-09-11T08:03:16.339 INFO:teuthology.orchestra.run.smithi122.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T08:03:16.339 INFO:teuthology.orchestra.run.smithi122.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T08:03:16.339 INFO:teuthology.orchestra.run.smithi122.stdout:Access: 2024-09-11 08:03:01.193468390 +0000 2024-09-11T08:03:16.339 INFO:teuthology.orchestra.run.smithi122.stdout:Modify: 2024-09-11 07:58:18.617876352 +0000 2024-09-11T08:03:16.339 INFO:teuthology.orchestra.run.smithi122.stdout:Change: 2024-09-11 07:58:18.617876352 +0000 2024-09-11T08:03:16.339 INFO:teuthology.orchestra.run.smithi122.stdout: Birth: 2024-09-11 07:58:18.617876352 +0000 2024-09-11T08:03:16.340 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-11T08:03:16.407 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records in 2024-09-11T08:03:16.407 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records out 2024-09-11T08:03:16.407 INFO:teuthology.orchestra.run.smithi122.stderr:512 bytes copied, 0.000243992 s, 2.1 MB/s 2024-09-11T08:03:16.409 DEBUG:teuthology.orchestra.run.smithi122:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-11T08:03:16.468 DEBUG:teuthology.orchestra.run.smithi122:> stat /dev/vg_nvme/lv_3 2024-09-11T08:03:16.525 INFO:teuthology.orchestra.run.smithi122.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-11T08:03:16.525 INFO:teuthology.orchestra.run.smithi122.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T08:03:16.525 INFO:teuthology.orchestra.run.smithi122.stdout:Device: 5h/5d Inode: 977 Links: 1 2024-09-11T08:03:16.526 INFO:teuthology.orchestra.run.smithi122.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T08:03:16.526 INFO:teuthology.orchestra.run.smithi122.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T08:03:16.526 INFO:teuthology.orchestra.run.smithi122.stdout:Access: 2024-09-11 08:03:01.193468390 +0000 2024-09-11T08:03:16.526 INFO:teuthology.orchestra.run.smithi122.stdout:Modify: 2024-09-11 07:58:18.616876452 +0000 2024-09-11T08:03:16.526 INFO:teuthology.orchestra.run.smithi122.stdout:Change: 2024-09-11 07:58:18.616876452 +0000 2024-09-11T08:03:16.526 INFO:teuthology.orchestra.run.smithi122.stdout: Birth: 2024-09-11 07:58:18.616876452 +0000 2024-09-11T08:03:16.526 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-11T08:03:16.591 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records in 2024-09-11T08:03:16.591 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records out 2024-09-11T08:03:16.591 INFO:teuthology.orchestra.run.smithi122.stderr:512 bytes copied, 0.000224099 s, 2.3 MB/s 2024-09-11T08:03:16.592 DEBUG:teuthology.orchestra.run.smithi122:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-11T08:03:16.649 DEBUG:teuthology.orchestra.run.smithi122:> stat /dev/vg_nvme/lv_4 2024-09-11T08:03:16.706 INFO:teuthology.orchestra.run.smithi122.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-11T08:03:16.706 INFO:teuthology.orchestra.run.smithi122.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T08:03:16.706 INFO:teuthology.orchestra.run.smithi122.stdout:Device: 5h/5d Inode: 968 Links: 1 2024-09-11T08:03:16.706 INFO:teuthology.orchestra.run.smithi122.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T08:03:16.706 INFO:teuthology.orchestra.run.smithi122.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T08:03:16.707 INFO:teuthology.orchestra.run.smithi122.stdout:Access: 2024-09-11 08:03:01.193468390 +0000 2024-09-11T08:03:16.707 INFO:teuthology.orchestra.run.smithi122.stdout:Modify: 2024-09-11 07:58:18.615876553 +0000 2024-09-11T08:03:16.707 INFO:teuthology.orchestra.run.smithi122.stdout:Change: 2024-09-11 07:58:18.615876553 +0000 2024-09-11T08:03:16.707 INFO:teuthology.orchestra.run.smithi122.stdout: Birth: 2024-09-11 07:58:18.615876553 +0000 2024-09-11T08:03:16.707 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-11T08:03:16.771 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records in 2024-09-11T08:03:16.771 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records out 2024-09-11T08:03:16.771 INFO:teuthology.orchestra.run.smithi122.stderr:512 bytes copied, 0.000222435 s, 2.3 MB/s 2024-09-11T08:03:16.772 DEBUG:teuthology.orchestra.run.smithi122:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-11T08:03:16.829 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-09-11T08:03:16.829 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/scratch_devs of=/dev/stdout 2024-09-11T08:03:16.846 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-11T08:03:16.846 DEBUG:teuthology.orchestra.run.smithi135:> stat /dev/vg_nvme/lv_1 2024-09-11T08:03:16.902 INFO:teuthology.orchestra.run.smithi135.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-11T08:03:16.902 INFO:teuthology.orchestra.run.smithi135.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T08:03:16.902 INFO:teuthology.orchestra.run.smithi135.stdout:Device: 5h/5d Inode: 977 Links: 1 2024-09-11T08:03:16.902 INFO:teuthology.orchestra.run.smithi135.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T08:03:16.903 INFO:teuthology.orchestra.run.smithi135.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T08:03:16.903 INFO:teuthology.orchestra.run.smithi135.stdout:Access: 2024-09-11 08:02:52.387730474 +0000 2024-09-11T08:03:16.903 INFO:teuthology.orchestra.run.smithi135.stdout:Modify: 2024-09-11 07:58:18.653713721 +0000 2024-09-11T08:03:16.903 INFO:teuthology.orchestra.run.smithi135.stdout:Change: 2024-09-11 07:58:18.653713721 +0000 2024-09-11T08:03:16.903 INFO:teuthology.orchestra.run.smithi135.stdout: Birth: 2024-09-11 07:58:18.653713721 +0000 2024-09-11T08:03:16.903 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-11T08:03:16.967 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records in 2024-09-11T08:03:16.967 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records out 2024-09-11T08:03:16.967 INFO:teuthology.orchestra.run.smithi135.stderr:512 bytes copied, 0.000215927 s, 2.4 MB/s 2024-09-11T08:03:16.968 DEBUG:teuthology.orchestra.run.smithi135:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-11T08:03:17.026 DEBUG:teuthology.orchestra.run.smithi135:> stat /dev/vg_nvme/lv_2 2024-09-11T08:03:17.085 INFO:teuthology.orchestra.run.smithi135.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-11T08:03:17.085 INFO:teuthology.orchestra.run.smithi135.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T08:03:17.085 INFO:teuthology.orchestra.run.smithi135.stdout:Device: 5h/5d Inode: 970 Links: 1 2024-09-11T08:03:17.085 INFO:teuthology.orchestra.run.smithi135.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T08:03:17.085 INFO:teuthology.orchestra.run.smithi135.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T08:03:17.085 INFO:teuthology.orchestra.run.smithi135.stdout:Access: 2024-09-11 08:02:52.388730453 +0000 2024-09-11T08:03:17.085 INFO:teuthology.orchestra.run.smithi135.stdout:Modify: 2024-09-11 07:58:18.653713721 +0000 2024-09-11T08:03:17.086 INFO:teuthology.orchestra.run.smithi135.stdout:Change: 2024-09-11 07:58:18.653713721 +0000 2024-09-11T08:03:17.086 INFO:teuthology.orchestra.run.smithi135.stdout: Birth: 2024-09-11 07:58:18.653713721 +0000 2024-09-11T08:03:17.086 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-11T08:03:17.191 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:16 smithi135 ceph-mon[32689]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi122=y;smithi135=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:03:17.192 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:16 smithi135 ceph-mon[32689]: Saving service mgr spec with placement smithi122=y;smithi135=x;count:2 2024-09-11T08:03:17.192 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:16 smithi135 ceph-mon[32689]: Deploying daemon mgr.x on smithi135 2024-09-11T08:03:17.193 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records in 2024-09-11T08:03:17.193 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records out 2024-09-11T08:03:17.193 INFO:teuthology.orchestra.run.smithi135.stderr:512 bytes copied, 0.000190265 s, 2.7 MB/s 2024-09-11T08:03:17.194 DEBUG:teuthology.orchestra.run.smithi135:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-11T08:03:17.210 DEBUG:teuthology.orchestra.run.smithi135:> stat /dev/vg_nvme/lv_3 2024-09-11T08:03:17.234 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:16 smithi122 ceph-mon[28992]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi122=y;smithi135=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:03:17.234 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:16 smithi122 ceph-mon[28992]: Saving service mgr spec with placement smithi122=y;smithi135=x;count:2 2024-09-11T08:03:17.234 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:16 smithi122 ceph-mon[28992]: Deploying daemon mgr.x on smithi135 2024-09-11T08:03:17.234 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:16 smithi122 ceph-mon[39726]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi122=y;smithi135=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:03:17.234 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:16 smithi122 ceph-mon[39726]: Saving service mgr spec with placement smithi122=y;smithi135=x;count:2 2024-09-11T08:03:17.234 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:16 smithi122 ceph-mon[39726]: Deploying daemon mgr.x on smithi135 2024-09-11T08:03:17.271 INFO:teuthology.orchestra.run.smithi135.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-11T08:03:17.271 INFO:teuthology.orchestra.run.smithi135.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T08:03:17.271 INFO:teuthology.orchestra.run.smithi135.stdout:Device: 5h/5d Inode: 982 Links: 1 2024-09-11T08:03:17.271 INFO:teuthology.orchestra.run.smithi135.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T08:03:17.271 INFO:teuthology.orchestra.run.smithi135.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T08:03:17.271 INFO:teuthology.orchestra.run.smithi135.stdout:Access: 2024-09-11 08:03:17.144202849 +0000 2024-09-11T08:03:17.271 INFO:teuthology.orchestra.run.smithi135.stdout:Modify: 2024-09-11 07:58:18.655713293 +0000 2024-09-11T08:03:17.271 INFO:teuthology.orchestra.run.smithi135.stdout:Change: 2024-09-11 07:58:18.655713293 +0000 2024-09-11T08:03:17.272 INFO:teuthology.orchestra.run.smithi135.stdout: Birth: 2024-09-11 07:58:18.655713293 +0000 2024-09-11T08:03:17.272 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-11T08:03:17.380 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records in 2024-09-11T08:03:17.380 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records out 2024-09-11T08:03:17.380 INFO:teuthology.orchestra.run.smithi135.stderr:512 bytes copied, 0.000217505 s, 2.4 MB/s 2024-09-11T08:03:17.382 DEBUG:teuthology.orchestra.run.smithi135:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-11T08:03:17.397 DEBUG:teuthology.orchestra.run.smithi135:> stat /dev/vg_nvme/lv_4 2024-09-11T08:03:17.454 INFO:teuthology.orchestra.run.smithi135.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-11T08:03:17.454 INFO:teuthology.orchestra.run.smithi135.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T08:03:17.454 INFO:teuthology.orchestra.run.smithi135.stdout:Device: 5h/5d Inode: 957 Links: 1 2024-09-11T08:03:17.454 INFO:teuthology.orchestra.run.smithi135.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T08:03:17.454 INFO:teuthology.orchestra.run.smithi135.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T08:03:17.455 INFO:teuthology.orchestra.run.smithi135.stdout:Access: 2024-09-11 08:03:17.336198757 +0000 2024-09-11T08:03:17.455 INFO:teuthology.orchestra.run.smithi135.stdout:Modify: 2024-09-11 07:58:18.651714149 +0000 2024-09-11T08:03:17.455 INFO:teuthology.orchestra.run.smithi135.stdout:Change: 2024-09-11 07:58:18.651714149 +0000 2024-09-11T08:03:17.455 INFO:teuthology.orchestra.run.smithi135.stdout: Birth: 2024-09-11 07:58:18.651714149 +0000 2024-09-11T08:03:17.455 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-11T08:03:17.478 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:03:17 smithi135 systemd[1]: Starting Ceph mgr.x for f1d75850-7013-11ef-bcea-c7b262605968... 2024-09-11T08:03:17.503 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records in 2024-09-11T08:03:17.503 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records out 2024-09-11T08:03:17.503 INFO:teuthology.orchestra.run.smithi135.stderr:512 bytes copied, 0.000228019 s, 2.2 MB/s 2024-09-11T08:03:17.504 DEBUG:teuthology.orchestra.run.smithi135:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-11T08:03:17.561 INFO:tasks.cephadm:Deploying osd.0 on smithi122 with /dev/vg_nvme/lv_4... 2024-09-11T08:03:17.561 DEBUG:teuthology.orchestra.run.smithi122:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f1d75850-7013-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-09-11T08:03:17.762 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:03:17 smithi135 podman[34518]: 2024-09-11 08:03:17.517430348 +0000 UTC m=+0.015771174 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T08:03:17.762 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:03:17 smithi135 podman[34518]: 2024-09-11 08:03:17.658460905 +0000 UTC m=+0.156801723 container create 4db7f0d5daf8471d85eb45d8f754d389b2724e732ce9ba2c592cc3352a3f6fe2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T08:03:17.817 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:03:18.197 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:17 smithi122 ceph-mon[28992]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:03:18.197 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:17 smithi122 ceph-mon[39726]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:03:18.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:17 smithi135 ceph-mon[32689]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:03:18.228 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:03:17 smithi135 podman[34518]: 2024-09-11 08:03:17.902708128 +0000 UTC m=+0.401048944 container init 4db7f0d5daf8471d85eb45d8f754d389b2724e732ce9ba2c592cc3352a3f6fe2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828) 2024-09-11T08:03:18.228 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:03:17 smithi135 podman[34518]: 2024-09-11 08:03:17.905481017 +0000 UTC m=+0.403821838 container start 4db7f0d5daf8471d85eb45d8f754d389b2724e732ce9ba2c592cc3352a3f6fe2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, ceph=True) 2024-09-11T08:03:18.228 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:03:17 smithi135 ceph-mgr[34558]: -- 172.21.15.135:0/194448661 <== mon.1 v2:172.21.15.135:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55eb1c3dc340 con 0x55eb1c3e8c00 2024-09-11T08:03:18.228 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:03:17 smithi135 bash[34518]: 4db7f0d5daf8471d85eb45d8f754d389b2724e732ce9ba2c592cc3352a3f6fe2 2024-09-11T08:03:18.228 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:03:17 smithi135 systemd[1]: Started Ceph mgr.x for f1d75850-7013-11ef-bcea-c7b262605968. 2024-09-11T08:03:18.229 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:03:18 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:03:18.020+0000 7f0e50fb6ec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T08:03:18.229 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:03:18 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:03:18.078+0000 7f0e50fb6ec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T08:03:18.229 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:03:18 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:03:18.145+0000 7f0e50fb6ec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T08:03:18.704 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:03:18 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:03:18.275+0000 7f0e50fb6ec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T08:03:18.704 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:03:18 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:03:18.431+0000 7f0e50fb6ec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T08:03:18.704 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:03:18 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:03:18.702+0000 7f0e50fb6ec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T08:03:18.978 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:03:18 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:03:18.765+0000 7f0e50fb6ec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T08:03:18.979 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:03:18 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:03:18.937+0000 7f0e50fb6ec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T08:03:19.180 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:19 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:19.180 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:19 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:19.180 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:19 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:19.180 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:19 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:19.181 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:19 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:03:19.181 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:19 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:19.181 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:19 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:19.181 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:19 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:19.181 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:19 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:19.181 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:19 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:03:19.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:19 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:19.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:19 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:19.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:19 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:19.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:19 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:19.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:19 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:03:19.559 INFO:teuthology.orchestra.run.smithi122.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-09-11T08:03:19.655 INFO:teuthology.orchestra.run.smithi122.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-09-11T08:03:19.670 INFO:teuthology.orchestra.run.smithi122.stderr: stderr: 10+0 records in 2024-09-11T08:03:19.670 INFO:teuthology.orchestra.run.smithi122.stderr:10+0 records out 2024-09-11T08:03:19.670 INFO:teuthology.orchestra.run.smithi122.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0129145 s, 812 MB/s 2024-09-11T08:03:19.670 INFO:teuthology.orchestra.run.smithi122.stderr:--> Zapping successful for: 2024-09-11T08:03:20.181 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:03:19 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:03:19.797+0000 7f0e50fb6ec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T08:03:20.182 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:03:19 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:03:19.860+0000 7f0e50fb6ec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T08:03:20.429 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:20 smithi122 ceph-mon[39726]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:03:20.429 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:20 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:20.429 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:20 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:20.429 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:20 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:03:20.430 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:20 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:03:20.430 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:20 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:20.430 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:20 smithi122 ceph-mon[39726]: Reconfiguring mgr.y (unknown last config time)... 2024-09-11T08:03:20.430 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:20 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T08:03:20.430 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:20 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T08:03:20.430 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:20 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:03:20.430 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:20 smithi122 ceph-mon[39726]: Reconfiguring daemon mgr.y on smithi122 2024-09-11T08:03:20.430 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:20 smithi122 ceph-mon[28992]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:03:20.430 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:20 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:20.430 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:20 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:20.430 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:20 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:03:20.430 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:20 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:03:20.431 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:20 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:20.431 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:20 smithi122 ceph-mon[28992]: Reconfiguring mgr.y (unknown last config time)... 2024-09-11T08:03:20.431 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:20 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T08:03:20.431 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:20 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T08:03:20.431 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:20 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:03:20.431 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:20 smithi122 ceph-mon[28992]: Reconfiguring daemon mgr.y on smithi122 2024-09-11T08:03:20.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:20 smithi135 ceph-mon[32689]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:03:20.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:20 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:20.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:20 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:20.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:20 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:03:20.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:20 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:03:20.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:20 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:20.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:20 smithi135 ceph-mon[32689]: Reconfiguring mgr.y (unknown last config time)... 2024-09-11T08:03:20.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:20 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T08:03:20.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:20 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T08:03:20.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:20 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:03:20.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:20 smithi135 ceph-mon[32689]: Reconfiguring daemon mgr.y on smithi122 2024-09-11T08:03:20.729 DEBUG:teuthology.orchestra.run.smithi122:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f1d75850-7013-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi122:vg_nvme/lv_4 2024-09-11T08:03:20.862 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:03:20 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:03:20.572+0000 7f0e50fb6ec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T08:03:20.862 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:03:20 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:03:20.681+0000 7f0e50fb6ec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T08:03:20.862 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:03:20 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:03:20.801+0000 7f0e50fb6ec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T08:03:20.862 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:03:20 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:03:20.860+0000 7f0e50fb6ec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T08:03:21.114 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:03:20 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:03:20.987+0000 7f0e50fb6ec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T08:03:21.114 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:03:21 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:03:21.054+0000 7f0e50fb6ec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T08:03:21.114 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:03:21 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:03:21.112+0000 7f0e50fb6ec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T08:03:21.310 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:03:21.379 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:03:21 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:03:21.246+0000 7f0e50fb6ec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T08:03:21.379 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:03:21 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:03:21.310+0000 7f0e50fb6ec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T08:03:21.379 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:03:21 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:03:21.377+0000 7f0e50fb6ec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T08:03:21.697 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:03:21 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:03:21.457+0000 7f0e50fb6ec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T08:03:21.698 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:03:21 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:03:21.515+0000 7f0e50fb6ec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T08:03:21.698 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:03:21 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:03:21.571+0000 7f0e50fb6ec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T08:03:21.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:21 smithi135 ceph-mon[32689]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:03:21.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:21 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:21.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:21 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:21.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:21 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:21.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:21 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:03:21.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:21 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:03:21.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:21 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:03:21.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:21 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:21.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:21 smithi122 ceph-mon[39726]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:03:21.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:21 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:21.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:21 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:21.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:21 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:21.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:21 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:03:21.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:21 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:03:21.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:21 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:03:21.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:21 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:21.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:21 smithi122 ceph-mon[28992]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:03:21.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:21 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:21.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:21 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:21.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:21 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:21.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:21 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:03:21.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:21 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:03:21.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:21 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:03:21.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:21 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:22.702 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:03:22 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:03:22.311+0000 7f0e50fb6ec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T08:03:22.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:22 smithi135 ceph-mon[32689]: Standby manager daemon x started 2024-09-11T08:03:22.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:22 smithi135 ceph-mon[32689]: from='mgr.? 172.21.15.135:0/740559514' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-11T08:03:22.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:22 smithi135 ceph-mon[32689]: from='mgr.? 172.21.15.135:0/740559514' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T08:03:22.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:22 smithi135 ceph-mon[32689]: from='mgr.? 172.21.15.135:0/740559514' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-11T08:03:22.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:22 smithi135 ceph-mon[32689]: from='mgr.? 172.21.15.135:0/740559514' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T08:03:22.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:22 smithi122 ceph-mon[28992]: Standby manager daemon x started 2024-09-11T08:03:22.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:22 smithi122 ceph-mon[28992]: from='mgr.? 172.21.15.135:0/740559514' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-11T08:03:22.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:22 smithi122 ceph-mon[28992]: from='mgr.? 172.21.15.135:0/740559514' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T08:03:22.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:22 smithi122 ceph-mon[28992]: from='mgr.? 172.21.15.135:0/740559514' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-11T08:03:22.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:22 smithi122 ceph-mon[28992]: from='mgr.? 172.21.15.135:0/740559514' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T08:03:22.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:22 smithi122 ceph-mon[39726]: Standby manager daemon x started 2024-09-11T08:03:22.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:22 smithi122 ceph-mon[39726]: from='mgr.? 172.21.15.135:0/740559514' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-11T08:03:22.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:22 smithi122 ceph-mon[39726]: from='mgr.? 172.21.15.135:0/740559514' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T08:03:22.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:22 smithi122 ceph-mon[39726]: from='mgr.? 172.21.15.135:0/740559514' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-11T08:03:22.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:22 smithi122 ceph-mon[39726]: from='mgr.? 172.21.15.135:0/740559514' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T08:03:23.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:23 smithi135 ceph-mon[32689]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:03:23.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:23 smithi135 ceph-mon[32689]: mgrmap e14: y(active, since 62s), standbys: x 2024-09-11T08:03:23.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:23 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T08:03:23.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:23 smithi135 ceph-mon[32689]: from='client.24119 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:03:23.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:23 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:03:23.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:23 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:03:23.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:23 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:03:23.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:23 smithi122 ceph-mon[39726]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:03:23.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:23 smithi122 ceph-mon[39726]: mgrmap e14: y(active, since 62s), standbys: x 2024-09-11T08:03:23.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:23 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T08:03:23.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:23 smithi122 ceph-mon[39726]: from='client.24119 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:03:23.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:23 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:03:23.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:23 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:03:23.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:23 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:03:23.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:23 smithi122 ceph-mon[28992]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:03:23.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:23 smithi122 ceph-mon[28992]: mgrmap e14: y(active, since 62s), standbys: x 2024-09-11T08:03:23.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:23 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T08:03:23.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:23 smithi122 ceph-mon[28992]: from='client.24119 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:03:23.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:23 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:03:23.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:23 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:03:23.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:23 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:03:25.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:25 smithi135 ceph-mon[32689]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:03:25.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:25 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/3434111029' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "17a779cc-5769-4022-a66e-e4ef151dd3ba"}]: dispatch 2024-09-11T08:03:25.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:25 smithi135 ceph-mon[32689]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "17a779cc-5769-4022-a66e-e4ef151dd3ba"}]: dispatch 2024-09-11T08:03:25.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:25 smithi135 ceph-mon[32689]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "17a779cc-5769-4022-a66e-e4ef151dd3ba"}]': finished 2024-09-11T08:03:25.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:25 smithi135 ceph-mon[32689]: osdmap e5: 1 total, 0 up, 1 in 2024-09-11T08:03:25.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:25 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:03:25.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:25 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/528824229' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:03:25.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:25 smithi122 ceph-mon[28992]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:03:25.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:25 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/3434111029' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "17a779cc-5769-4022-a66e-e4ef151dd3ba"}]: dispatch 2024-09-11T08:03:25.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:25 smithi122 ceph-mon[28992]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "17a779cc-5769-4022-a66e-e4ef151dd3ba"}]: dispatch 2024-09-11T08:03:25.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:25 smithi122 ceph-mon[28992]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "17a779cc-5769-4022-a66e-e4ef151dd3ba"}]': finished 2024-09-11T08:03:25.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:25 smithi122 ceph-mon[28992]: osdmap e5: 1 total, 0 up, 1 in 2024-09-11T08:03:25.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:25 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:03:25.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:25 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/528824229' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:03:25.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:25 smithi122 ceph-mon[39726]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:03:25.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:25 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/3434111029' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "17a779cc-5769-4022-a66e-e4ef151dd3ba"}]: dispatch 2024-09-11T08:03:25.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:25 smithi122 ceph-mon[39726]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "17a779cc-5769-4022-a66e-e4ef151dd3ba"}]: dispatch 2024-09-11T08:03:25.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:25 smithi122 ceph-mon[39726]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "17a779cc-5769-4022-a66e-e4ef151dd3ba"}]': finished 2024-09-11T08:03:25.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:25 smithi122 ceph-mon[39726]: osdmap e5: 1 total, 0 up, 1 in 2024-09-11T08:03:25.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:25 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:03:25.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:25 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/528824229' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:03:27.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:27 smithi135 ceph-mon[32689]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:03:27.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:27 smithi122 ceph-mon[28992]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:03:27.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:27 smithi122 ceph-mon[39726]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:03:29.850 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:29 smithi122 ceph-mon[28992]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:03:29.850 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:29 smithi122 ceph-mon[39726]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:03:29.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:29 smithi135 ceph-mon[32689]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:03:31.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:31 smithi135 ceph-mon[32689]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:03:31.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:31 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-11T08:03:31.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:31 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:03:31.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:31 smithi135 ceph-mon[32689]: Deploying daemon osd.0 on smithi122 2024-09-11T08:03:31.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:31 smithi122 ceph-mon[28992]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:03:31.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:31 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-11T08:03:31.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:31 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:03:31.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:31 smithi122 ceph-mon[28992]: Deploying daemon osd.0 on smithi122 2024-09-11T08:03:31.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:31 smithi122 ceph-mon[39726]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:03:31.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:31 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-11T08:03:31.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:31 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:03:31.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:31 smithi122 ceph-mon[39726]: Deploying daemon osd.0 on smithi122 2024-09-11T08:03:33.766 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:33 smithi122 ceph-mon[28992]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:03:33.767 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:33 smithi122 ceph-mon[39726]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:03:33.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:33 smithi135 ceph-mon[32689]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:03:35.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:35 smithi135 ceph-mon[32689]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:03:35.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:35 smithi122 ceph-mon[28992]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:03:35.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:35 smithi122 ceph-mon[39726]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:03:36.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:36 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:03:36.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:36 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:36.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:36 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:36.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:36 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:03:36.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:36 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:36.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:36 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:36.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:36 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:03:36.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:36 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:36.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:36 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:38.139 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:37 smithi122 ceph-mon[28992]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:03:38.140 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:37 smithi122 ceph-mon[39726]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:03:38.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:37 smithi135 ceph-mon[32689]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:03:39.209 INFO:teuthology.orchestra.run.smithi122.stdout:Created osd(s) 0 on host 'smithi122' 2024-09-11T08:03:39.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:39 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:39.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:39 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:39.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:39 smithi122 ceph-mon[28992]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:03:39.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:39 smithi122 ceph-mon[28992]: from='osd.0 [v2:172.21.15.122:6802/578396583,v1:172.21.15.122:6803/578396583]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-11T08:03:39.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:39 smithi122 ceph-mon[28992]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-11T08:03:39.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:39 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:39.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:39 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:39.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:39 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:39.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:39 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:39.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:39 smithi122 ceph-mon[39726]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:03:39.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:39 smithi122 ceph-mon[39726]: from='osd.0 [v2:172.21.15.122:6802/578396583,v1:172.21.15.122:6803/578396583]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-11T08:03:39.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:39 smithi122 ceph-mon[39726]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-11T08:03:39.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:39 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:39.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:39 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:39.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:39 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:39.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:39 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:39.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:39 smithi135 ceph-mon[32689]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:03:39.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:39 smithi135 ceph-mon[32689]: from='osd.0 [v2:172.21.15.122:6802/578396583,v1:172.21.15.122:6803/578396583]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-11T08:03:39.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:39 smithi135 ceph-mon[32689]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-11T08:03:39.729 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:39 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:39.729 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:39 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:39.813 DEBUG:teuthology.orchestra.run.smithi122:osd.0> sudo journalctl -f -n 0 -u ceph-f1d75850-7013-11ef-bcea-c7b262605968@osd.0.service 2024-09-11T08:03:39.816 INFO:tasks.cephadm:Deploying osd.1 on smithi122 with /dev/vg_nvme/lv_3... 2024-09-11T08:03:39.816 DEBUG:teuthology.orchestra.run.smithi122:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f1d75850-7013-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-09-11T08:03:40.106 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:03:40.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:40 smithi122 ceph-mon[39726]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-11T08:03:40.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:40 smithi122 ceph-mon[39726]: osdmap e6: 1 total, 0 up, 1 in 2024-09-11T08:03:40.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:40 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:03:40.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:40 smithi122 ceph-mon[39726]: from='osd.0 [v2:172.21.15.122:6802/578396583,v1:172.21.15.122:6803/578396583]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-09-11T08:03:40.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:40 smithi122 ceph-mon[39726]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-09-11T08:03:40.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:40 smithi122 ceph-mon[28992]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-11T08:03:40.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:40 smithi122 ceph-mon[28992]: osdmap e6: 1 total, 0 up, 1 in 2024-09-11T08:03:40.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:40 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:03:40.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:40 smithi122 ceph-mon[28992]: from='osd.0 [v2:172.21.15.122:6802/578396583,v1:172.21.15.122:6803/578396583]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-09-11T08:03:40.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:40 smithi122 ceph-mon[28992]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-09-11T08:03:40.485 INFO:journalctl@ceph.osd.0.smithi122.stdout:Sep 11 08:03:40 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-0[43596]: 2024-09-11T08:03:40.396+0000 7f9699133640 -1 osd.0 0 waiting for initial osdmap 2024-09-11T08:03:40.485 INFO:journalctl@ceph.osd.0.smithi122.stdout:Sep 11 08:03:40 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-0[43596]: 2024-09-11T08:03:40.399+0000 7f9693769640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T08:03:40.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:40 smithi135 ceph-mon[32689]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-11T08:03:40.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:40 smithi135 ceph-mon[32689]: osdmap e6: 1 total, 0 up, 1 in 2024-09-11T08:03:40.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:40 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:03:40.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:40 smithi135 ceph-mon[32689]: from='osd.0 [v2:172.21.15.122:6802/578396583,v1:172.21.15.122:6803/578396583]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-09-11T08:03:40.729 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:40 smithi135 ceph-mon[32689]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-09-11T08:03:41.541 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:41 smithi122 ceph-mon[28992]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-09-11T08:03:41.542 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:41 smithi122 ceph-mon[28992]: osdmap e7: 1 total, 0 up, 1 in 2024-09-11T08:03:41.542 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:41 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:03:41.542 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:41 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:03:41.542 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:41 smithi122 ceph-mon[28992]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:03:41.542 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:41 smithi122 ceph-mon[39726]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-09-11T08:03:41.542 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:41 smithi122 ceph-mon[39726]: osdmap e7: 1 total, 0 up, 1 in 2024-09-11T08:03:41.542 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:41 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:03:41.542 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:41 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:03:41.542 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:41 smithi122 ceph-mon[39726]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:03:41.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:41 smithi135 ceph-mon[32689]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-09-11T08:03:41.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:41 smithi135 ceph-mon[32689]: osdmap e7: 1 total, 0 up, 1 in 2024-09-11T08:03:41.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:41 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:03:41.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:41 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:03:41.729 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:41 smithi135 ceph-mon[32689]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:03:42.085 INFO:teuthology.orchestra.run.smithi122.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-09-11T08:03:42.166 INFO:teuthology.orchestra.run.smithi122.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-09-11T08:03:42.185 INFO:teuthology.orchestra.run.smithi122.stderr: stderr: 10+0 records in 2024-09-11T08:03:42.185 INFO:teuthology.orchestra.run.smithi122.stderr:10+0 records out 2024-09-11T08:03:42.185 INFO:teuthology.orchestra.run.smithi122.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0166829 s, 629 MB/s 2024-09-11T08:03:42.185 INFO:teuthology.orchestra.run.smithi122.stderr:--> Zapping successful for: 2024-09-11T08:03:42.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:42 smithi122 ceph-mon[28992]: purged_snaps scrub starts 2024-09-11T08:03:42.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:42 smithi122 ceph-mon[28992]: purged_snaps scrub ok 2024-09-11T08:03:42.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:42 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:03:42.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:42 smithi122 ceph-mon[28992]: osd.0 [v2:172.21.15.122:6802/578396583,v1:172.21.15.122:6803/578396583] boot 2024-09-11T08:03:42.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:42 smithi122 ceph-mon[28992]: osdmap e8: 1 total, 1 up, 1 in 2024-09-11T08:03:42.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:42 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:03:42.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:42 smithi122 ceph-mon[39726]: purged_snaps scrub starts 2024-09-11T08:03:42.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:42 smithi122 ceph-mon[39726]: purged_snaps scrub ok 2024-09-11T08:03:42.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:42 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:03:42.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:42 smithi122 ceph-mon[39726]: osd.0 [v2:172.21.15.122:6802/578396583,v1:172.21.15.122:6803/578396583] boot 2024-09-11T08:03:42.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:42 smithi122 ceph-mon[39726]: osdmap e8: 1 total, 1 up, 1 in 2024-09-11T08:03:42.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:42 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:03:42.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:42 smithi135 ceph-mon[32689]: purged_snaps scrub starts 2024-09-11T08:03:42.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:42 smithi135 ceph-mon[32689]: purged_snaps scrub ok 2024-09-11T08:03:42.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:42 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:03:42.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:42 smithi135 ceph-mon[32689]: osd.0 [v2:172.21.15.122:6802/578396583,v1:172.21.15.122:6803/578396583] boot 2024-09-11T08:03:42.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:42 smithi135 ceph-mon[32689]: osdmap e8: 1 total, 1 up, 1 in 2024-09-11T08:03:42.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:42 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:03:43.545 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:43 smithi122 ceph-mon[28992]: osdmap e9: 1 total, 1 up, 1 in 2024-09-11T08:03:43.546 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:43 smithi122 ceph-mon[28992]: pgmap v40: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:03:43.546 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:43 smithi122 ceph-mon[39726]: osdmap e9: 1 total, 1 up, 1 in 2024-09-11T08:03:43.546 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:43 smithi122 ceph-mon[39726]: pgmap v40: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:03:43.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:43 smithi135 ceph-mon[32689]: osdmap e9: 1 total, 1 up, 1 in 2024-09-11T08:03:43.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:43 smithi135 ceph-mon[32689]: pgmap v40: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:03:43.748 DEBUG:teuthology.orchestra.run.smithi122:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f1d75850-7013-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi122:vg_nvme/lv_3 2024-09-11T08:03:44.619 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:03:45.862 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:45 smithi122 ceph-mon[28992]: Detected new or changed devices on smithi122 2024-09-11T08:03:45.863 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:45 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:45.863 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:45 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:45.863 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:45 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:03:45.863 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:45 smithi122 ceph-mon[28992]: Adjusting osd_memory_target on smithi122 to 16063M 2024-09-11T08:03:45.863 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:45 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:45.863 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:45 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:03:45.863 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:45 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:03:45.863 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:45 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:45.863 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:45 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:03:45.863 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:45 smithi122 ceph-mon[28992]: pgmap v41: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:03:45.863 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:45 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:03:45.863 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:45 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:03:45.864 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:45 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:45.864 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:45 smithi122 ceph-mon[39726]: Detected new or changed devices on smithi122 2024-09-11T08:03:45.864 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:45 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:45.864 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:45 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:45.864 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:45 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:03:45.864 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:45 smithi122 ceph-mon[39726]: Adjusting osd_memory_target on smithi122 to 16063M 2024-09-11T08:03:45.864 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:45 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:45.864 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:45 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:03:45.864 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:45 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:03:45.864 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:45 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:45.864 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:45 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:03:45.865 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:45 smithi122 ceph-mon[39726]: pgmap v41: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:03:45.865 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:45 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:03:45.865 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:45 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:03:45.865 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:45 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:45.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:45 smithi135 ceph-mon[32689]: Detected new or changed devices on smithi122 2024-09-11T08:03:45.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:45 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:45.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:45 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:45.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:45 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:03:45.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:45 smithi135 ceph-mon[32689]: Adjusting osd_memory_target on smithi122 to 16063M 2024-09-11T08:03:45.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:45 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:45.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:45 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:03:45.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:45 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:03:45.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:45 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:45.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:45 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:03:45.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:45 smithi135 ceph-mon[32689]: pgmap v41: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:03:45.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:45 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:03:45.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:45 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:03:45.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:45 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:03:46.869 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:46 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:03:46.869 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:46 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:03:46.869 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:46 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:03:46.869 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:46 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:03:46.870 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:46 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:03:46.870 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:46 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:03:46.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:46 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:03:46.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:46 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:03:46.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:46 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:03:47.727 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:47 smithi122 ceph-mon[28992]: from='client.14238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:03:47.728 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:47 smithi122 ceph-mon[28992]: pgmap v42: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:03:47.728 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:47 smithi122 ceph-mon[39726]: from='client.14238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:03:47.728 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:47 smithi122 ceph-mon[39726]: pgmap v42: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:03:47.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:47 smithi135 ceph-mon[32689]: from='client.14238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:03:47.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:47 smithi135 ceph-mon[32689]: pgmap v42: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:03:48.939 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:48 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/971107923' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7b89946c-b41a-4c21-ba99-69cf6c474d1e"}]: dispatch 2024-09-11T08:03:48.939 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:48 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/971107923' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7b89946c-b41a-4c21-ba99-69cf6c474d1e"}]': finished 2024-09-11T08:03:48.940 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:48 smithi122 ceph-mon[28992]: osdmap e10: 2 total, 1 up, 2 in 2024-09-11T08:03:48.940 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:48 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:03:48.940 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:48 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/971107923' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7b89946c-b41a-4c21-ba99-69cf6c474d1e"}]: dispatch 2024-09-11T08:03:48.940 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:48 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/971107923' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7b89946c-b41a-4c21-ba99-69cf6c474d1e"}]': finished 2024-09-11T08:03:48.940 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:48 smithi122 ceph-mon[39726]: osdmap e10: 2 total, 1 up, 2 in 2024-09-11T08:03:48.940 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:48 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:03:48.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:48 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/971107923' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7b89946c-b41a-4c21-ba99-69cf6c474d1e"}]: dispatch 2024-09-11T08:03:48.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:48 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/971107923' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7b89946c-b41a-4c21-ba99-69cf6c474d1e"}]': finished 2024-09-11T08:03:48.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:48 smithi135 ceph-mon[32689]: osdmap e10: 2 total, 1 up, 2 in 2024-09-11T08:03:48.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:48 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:03:49.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:49 smithi135 ceph-mon[32689]: pgmap v44: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:03:49.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:49 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/1751409488' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:03:49.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:49 smithi122 ceph-mon[28992]: pgmap v44: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:03:49.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:49 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/1751409488' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:03:49.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:49 smithi122 ceph-mon[39726]: pgmap v44: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:03:49.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:49 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/1751409488' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:03:51.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:51 smithi135 ceph-mon[32689]: pgmap v45: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:03:51.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:51 smithi122 ceph-mon[28992]: pgmap v45: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:03:51.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:51 smithi122 ceph-mon[39726]: pgmap v45: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:03:53.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:53 smithi135 ceph-mon[32689]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:03:53.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:53 smithi122 ceph-mon[28992]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:03:53.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:53 smithi122 ceph-mon[39726]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:03:54.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:54 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-11T08:03:54.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:54 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:03:54.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:54 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-11T08:03:54.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:54 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:03:55.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:54 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-11T08:03:55.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:54 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:03:56.041 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:55 smithi122 ceph-mon[28992]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:03:56.041 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:55 smithi122 ceph-mon[28992]: Deploying daemon osd.1 on smithi122 2024-09-11T08:03:56.041 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:55 smithi122 ceph-mon[39726]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:03:56.041 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:55 smithi122 ceph-mon[39726]: Deploying daemon osd.1 on smithi122 2024-09-11T08:03:56.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:55 smithi135 ceph-mon[32689]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:03:56.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:55 smithi135 ceph-mon[32689]: Deploying daemon osd.1 on smithi122 2024-09-11T08:03:58.053 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:57 smithi122 ceph-mon[28992]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:03:58.053 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:57 smithi122 ceph-mon[39726]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:03:58.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:57 smithi135 ceph-mon[32689]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:04:00.010 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:59 smithi122 ceph-mon[39726]: pgmap v49: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:04:00.010 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:59 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:04:00.010 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:59 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:00.010 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:03:59 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:00.010 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:59 smithi122 ceph-mon[28992]: pgmap v49: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:04:00.010 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:59 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:04:00.010 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:59 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:00.010 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:03:59 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:00.030 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:59 smithi135 ceph-mon[32689]: pgmap v49: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:04:00.030 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:59 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:04:00.030 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:59 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:00.030 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:03:59 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:02.124 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:01 smithi122 ceph-mon[28992]: pgmap v50: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:04:02.124 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:01 smithi122 ceph-mon[39726]: pgmap v50: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:04:02.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:01 smithi135 ceph-mon[32689]: pgmap v50: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:04:02.346 INFO:teuthology.orchestra.run.smithi122.stdout:Created osd(s) 1 on host 'smithi122' 2024-09-11T08:04:03.002 DEBUG:teuthology.orchestra.run.smithi122:osd.1> sudo journalctl -f -n 0 -u ceph-f1d75850-7013-11ef-bcea-c7b262605968@osd.1.service 2024-09-11T08:04:03.004 INFO:tasks.cephadm:Deploying osd.2 on smithi122 with /dev/vg_nvme/lv_2... 2024-09-11T08:04:03.004 DEBUG:teuthology.orchestra.run.smithi122:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f1d75850-7013-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-09-11T08:04:03.294 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:04:03.697 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:03 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:03.697 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:03 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:03.697 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:03 smithi122 ceph-mon[28992]: from='osd.1 [v2:172.21.15.122:6810/3517945576,v1:172.21.15.122:6811/3517945576]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-11T08:04:03.697 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:03 smithi122 ceph-mon[28992]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-11T08:04:03.697 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:03 smithi122 ceph-mon[28992]: pgmap v51: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:04:03.697 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:03 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:03.698 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:03 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:03.698 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:03 smithi122 ceph-mon[39726]: from='osd.1 [v2:172.21.15.122:6810/3517945576,v1:172.21.15.122:6811/3517945576]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-11T08:04:03.698 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:03 smithi122 ceph-mon[39726]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-11T08:04:03.698 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:03 smithi122 ceph-mon[39726]: pgmap v51: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:04:03.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:03 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:03.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:03 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:03.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:03 smithi135 ceph-mon[32689]: from='osd.1 [v2:172.21.15.122:6810/3517945576,v1:172.21.15.122:6811/3517945576]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-11T08:04:03.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:03 smithi135 ceph-mon[32689]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-11T08:04:03.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:03 smithi135 ceph-mon[32689]: pgmap v51: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:04:04.434 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:04 smithi122 ceph-mon[28992]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-11T08:04:04.435 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:04 smithi122 ceph-mon[28992]: osdmap e11: 2 total, 1 up, 2 in 2024-09-11T08:04:04.435 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:04 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:04:04.435 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:04 smithi122 ceph-mon[28992]: from='osd.1 [v2:172.21.15.122:6810/3517945576,v1:172.21.15.122:6811/3517945576]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-09-11T08:04:04.435 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:04 smithi122 ceph-mon[28992]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-09-11T08:04:04.435 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:04 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:04.435 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:04 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:04.435 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:04 smithi122 ceph-mon[39726]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-11T08:04:04.436 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:04 smithi122 ceph-mon[39726]: osdmap e11: 2 total, 1 up, 2 in 2024-09-11T08:04:04.436 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:04 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:04:04.436 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:04 smithi122 ceph-mon[39726]: from='osd.1 [v2:172.21.15.122:6810/3517945576,v1:172.21.15.122:6811/3517945576]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-09-11T08:04:04.436 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:04 smithi122 ceph-mon[39726]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-09-11T08:04:04.436 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:04 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:04.436 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:04 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:04.436 INFO:journalctl@ceph.osd.1.smithi122.stdout:Sep 11 08:04:04 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-1[49473]: 2024-09-11T08:04:04.361+0000 7f5870ac1640 -1 osd.1 0 waiting for initial osdmap 2024-09-11T08:04:04.436 INFO:journalctl@ceph.osd.1.smithi122.stdout:Sep 11 08:04:04 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-1[49473]: 2024-09-11T08:04:04.363+0000 7f586a8f6640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T08:04:04.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:04 smithi135 ceph-mon[32689]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-11T08:04:04.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:04 smithi135 ceph-mon[32689]: osdmap e11: 2 total, 1 up, 2 in 2024-09-11T08:04:04.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:04 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:04:04.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:04 smithi135 ceph-mon[32689]: from='osd.1 [v2:172.21.15.122:6810/3517945576,v1:172.21.15.122:6811/3517945576]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-09-11T08:04:04.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:04 smithi135 ceph-mon[32689]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-09-11T08:04:04.729 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:04 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:04.729 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:04 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:05.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:05 smithi122 ceph-mon[39726]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-09-11T08:04:05.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:05 smithi122 ceph-mon[39726]: osdmap e12: 2 total, 1 up, 2 in 2024-09-11T08:04:05.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:05 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:04:05.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:05 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:04:05.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:05 smithi122 ceph-mon[39726]: pgmap v54: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:04:05.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:05 smithi122 ceph-mon[28992]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-09-11T08:04:05.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:05 smithi122 ceph-mon[28992]: osdmap e12: 2 total, 1 up, 2 in 2024-09-11T08:04:05.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:05 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:04:05.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:05 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:04:05.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:05 smithi122 ceph-mon[28992]: pgmap v54: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:04:05.653 INFO:teuthology.orchestra.run.smithi122.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-09-11T08:04:05.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:05 smithi135 ceph-mon[32689]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-09-11T08:04:05.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:05 smithi135 ceph-mon[32689]: osdmap e12: 2 total, 1 up, 2 in 2024-09-11T08:04:05.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:05 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:04:05.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:05 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:04:05.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:05 smithi135 ceph-mon[32689]: pgmap v54: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:04:05.747 INFO:teuthology.orchestra.run.smithi122.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-09-11T08:04:05.764 INFO:teuthology.orchestra.run.smithi122.stderr: stderr: 10+0 records in 2024-09-11T08:04:05.764 INFO:teuthology.orchestra.run.smithi122.stderr:10+0 records out 2024-09-11T08:04:05.764 INFO:teuthology.orchestra.run.smithi122.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0149069 s, 703 MB/s 2024-09-11T08:04:05.764 INFO:teuthology.orchestra.run.smithi122.stderr:--> Zapping successful for: 2024-09-11T08:04:06.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:06 smithi122 ceph-mon[39726]: purged_snaps scrub starts 2024-09-11T08:04:06.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:06 smithi122 ceph-mon[39726]: purged_snaps scrub ok 2024-09-11T08:04:06.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:06 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:04:06.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:06 smithi122 ceph-mon[39726]: osd.1 [v2:172.21.15.122:6810/3517945576,v1:172.21.15.122:6811/3517945576] boot 2024-09-11T08:04:06.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:06 smithi122 ceph-mon[39726]: osdmap e13: 2 total, 2 up, 2 in 2024-09-11T08:04:06.468 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:06 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:04:06.468 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:06 smithi122 ceph-mon[28992]: purged_snaps scrub starts 2024-09-11T08:04:06.468 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:06 smithi122 ceph-mon[28992]: purged_snaps scrub ok 2024-09-11T08:04:06.468 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:06 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:04:06.468 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:06 smithi122 ceph-mon[28992]: osd.1 [v2:172.21.15.122:6810/3517945576,v1:172.21.15.122:6811/3517945576] boot 2024-09-11T08:04:06.468 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:06 smithi122 ceph-mon[28992]: osdmap e13: 2 total, 2 up, 2 in 2024-09-11T08:04:06.468 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:06 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:04:06.490 DEBUG:teuthology.orchestra.run.smithi122:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f1d75850-7013-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi122:vg_nvme/lv_2 2024-09-11T08:04:06.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:06 smithi135 ceph-mon[32689]: purged_snaps scrub starts 2024-09-11T08:04:06.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:06 smithi135 ceph-mon[32689]: purged_snaps scrub ok 2024-09-11T08:04:06.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:06 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:04:06.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:06 smithi135 ceph-mon[32689]: osd.1 [v2:172.21.15.122:6810/3517945576,v1:172.21.15.122:6811/3517945576] boot 2024-09-11T08:04:06.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:06 smithi135 ceph-mon[32689]: osdmap e13: 2 total, 2 up, 2 in 2024-09-11T08:04:06.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:06 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:04:06.952 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:04:07.446 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:07 smithi122 ceph-mon[28992]: osdmap e14: 2 total, 2 up, 2 in 2024-09-11T08:04:07.446 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:07 smithi122 ceph-mon[28992]: pgmap v57: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:04:07.446 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:07 smithi122 ceph-mon[39726]: osdmap e14: 2 total, 2 up, 2 in 2024-09-11T08:04:07.446 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:07 smithi122 ceph-mon[39726]: pgmap v57: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:04:07.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:07 smithi135 ceph-mon[32689]: osdmap e14: 2 total, 2 up, 2 in 2024-09-11T08:04:07.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:07 smithi135 ceph-mon[32689]: pgmap v57: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:04:09.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:09 smithi135 ceph-mon[32689]: Detected new or changed devices on smithi122 2024-09-11T08:04:09.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:09 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:09.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:09 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:09.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:09 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:04:09.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:09 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:04:09.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:09 smithi135 ceph-mon[32689]: Adjusting osd_memory_target on smithi122 to 8031M 2024-09-11T08:04:09.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:09 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:09.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:09 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:04:09.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:09 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:04:09.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:09 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:09.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:09 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:04:09.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:09 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:04:09.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:09 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:04:09.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:09 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:09.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:09 smithi122 ceph-mon[28992]: Detected new or changed devices on smithi122 2024-09-11T08:04:09.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:09 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:09.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:09 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:09.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:09 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:04:09.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:09 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:04:09.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:09 smithi122 ceph-mon[28992]: Adjusting osd_memory_target on smithi122 to 8031M 2024-09-11T08:04:09.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:09 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:09.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:09 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:04:09.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:09 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:04:09.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:09 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:09.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:09 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:04:09.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:09 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:04:09.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:09 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:04:09.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:09 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:09.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:09 smithi122 ceph-mon[39726]: Detected new or changed devices on smithi122 2024-09-11T08:04:09.486 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:09 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:09.486 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:09 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:09.486 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:09 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:04:09.486 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:09 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:04:09.486 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:09 smithi122 ceph-mon[39726]: Adjusting osd_memory_target on smithi122 to 8031M 2024-09-11T08:04:09.486 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:09 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:09.486 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:09 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:04:09.486 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:09 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:04:09.486 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:09 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:09.486 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:09 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:04:09.486 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:09 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:04:09.486 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:09 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:04:09.487 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:09 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:10.234 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:10 smithi122 ceph-mon[28992]: pgmap v58: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:04:10.234 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:10 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:04:10.234 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:10 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:04:10.234 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:10 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:04:10.235 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:10 smithi122 ceph-mon[39726]: pgmap v58: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:04:10.235 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:10 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:04:10.235 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:10 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:04:10.235 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:10 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:04:10.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:10 smithi135 ceph-mon[32689]: pgmap v58: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:04:10.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:10 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:04:10.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:10 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:04:10.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:10 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:04:11.375 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:11 smithi122 ceph-mon[28992]: from='client.14265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:04:11.375 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:11 smithi122 ceph-mon[39726]: from='client.14265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:04:11.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:11 smithi135 ceph-mon[32689]: from='client.14265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:04:12.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:12 smithi135 ceph-mon[32689]: pgmap v59: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:04:12.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:12 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/3353457494' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aa9a050b-2829-4235-b379-60d327384fa1"}]: dispatch 2024-09-11T08:04:12.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:12 smithi135 ceph-mon[32689]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aa9a050b-2829-4235-b379-60d327384fa1"}]: dispatch 2024-09-11T08:04:12.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:12 smithi135 ceph-mon[32689]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aa9a050b-2829-4235-b379-60d327384fa1"}]': finished 2024-09-11T08:04:12.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:12 smithi135 ceph-mon[32689]: osdmap e15: 3 total, 2 up, 3 in 2024-09-11T08:04:12.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:12 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:04:12.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:12 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/1198694977' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:04:12.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:12 smithi122 ceph-mon[28992]: pgmap v59: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:04:12.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:12 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/3353457494' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aa9a050b-2829-4235-b379-60d327384fa1"}]: dispatch 2024-09-11T08:04:12.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:12 smithi122 ceph-mon[28992]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aa9a050b-2829-4235-b379-60d327384fa1"}]: dispatch 2024-09-11T08:04:12.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:12 smithi122 ceph-mon[28992]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aa9a050b-2829-4235-b379-60d327384fa1"}]': finished 2024-09-11T08:04:12.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:12 smithi122 ceph-mon[28992]: osdmap e15: 3 total, 2 up, 3 in 2024-09-11T08:04:12.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:12 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:04:12.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:12 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/1198694977' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:04:12.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:12 smithi122 ceph-mon[39726]: pgmap v59: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:04:12.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:12 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/3353457494' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aa9a050b-2829-4235-b379-60d327384fa1"}]: dispatch 2024-09-11T08:04:12.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:12 smithi122 ceph-mon[39726]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aa9a050b-2829-4235-b379-60d327384fa1"}]: dispatch 2024-09-11T08:04:12.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:12 smithi122 ceph-mon[39726]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aa9a050b-2829-4235-b379-60d327384fa1"}]': finished 2024-09-11T08:04:12.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:12 smithi122 ceph-mon[39726]: osdmap e15: 3 total, 2 up, 3 in 2024-09-11T08:04:12.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:12 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:04:12.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:12 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/1198694977' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:04:14.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:14 smithi122 ceph-mon[28992]: pgmap v61: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:04:14.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:14 smithi122 ceph-mon[39726]: pgmap v61: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:04:14.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:14 smithi135 ceph-mon[32689]: pgmap v61: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:04:16.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:16 smithi122 ceph-mon[28992]: pgmap v62: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:04:16.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:16 smithi122 ceph-mon[39726]: pgmap v62: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:04:16.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:16 smithi135 ceph-mon[32689]: pgmap v62: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:04:18.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:18 smithi122 ceph-mon[28992]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:04:18.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:18 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-11T08:04:18.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:18 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:04:18.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:18 smithi122 ceph-mon[39726]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:04:18.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:18 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-11T08:04:18.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:18 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:04:18.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:18 smithi135 ceph-mon[32689]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:04:18.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:18 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-11T08:04:18.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:18 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:04:19.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:19 smithi122 ceph-mon[28992]: Deploying daemon osd.2 on smithi122 2024-09-11T08:04:19.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:19 smithi122 ceph-mon[39726]: Deploying daemon osd.2 on smithi122 2024-09-11T08:04:19.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:19 smithi135 ceph-mon[32689]: Deploying daemon osd.2 on smithi122 2024-09-11T08:04:20.431 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:20 smithi122 ceph-mon[28992]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:04:20.432 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:20 smithi122 ceph-mon[39726]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:04:20.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:20 smithi135 ceph-mon[32689]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:04:22.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:22 smithi122 ceph-mon[28992]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:04:22.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:22 smithi122 ceph-mon[39726]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:04:22.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:22 smithi135 ceph-mon[32689]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:04:23.453 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:23 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:04:23.453 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:23 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:23.453 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:23 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:23.454 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:23 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:04:23.454 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:23 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:23.454 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:23 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:23.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:23 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:04:23.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:23 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:23.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:23 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:24.626 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:24 smithi122 ceph-mon[28992]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:04:24.626 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:24 smithi122 ceph-mon[39726]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:04:24.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:24 smithi135 ceph-mon[32689]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:04:26.174 INFO:teuthology.orchestra.run.smithi122.stdout:Created osd(s) 2 on host 'smithi122' 2024-09-11T08:04:26.478 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:26 smithi122 ceph-mon[39726]: pgmap v67: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:04:26.479 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:26 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:26.479 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:26 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:26.479 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:26 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:04:26.479 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:26 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:04:26.479 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:26 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:26.479 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:26 smithi122 ceph-mon[39726]: from='osd.2 [v2:172.21.15.122:6818/3927725903,v1:172.21.15.122:6819/3927725903]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-11T08:04:26.479 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:26 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:04:26.479 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:26 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:26.479 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:26 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:26.479 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:26 smithi122 ceph-mon[28992]: pgmap v67: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:04:26.479 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:26 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:26.480 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:26 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:26.480 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:26 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:04:26.480 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:26 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:04:26.480 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:26 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:26.480 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:26 smithi122 ceph-mon[28992]: from='osd.2 [v2:172.21.15.122:6818/3927725903,v1:172.21.15.122:6819/3927725903]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-11T08:04:26.480 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:26 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:04:26.480 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:26 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:26.480 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:26 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:26.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:26 smithi135 ceph-mon[32689]: pgmap v67: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:04:26.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:26 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:26.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:26 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:26.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:26 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:04:26.729 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:26 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:04:26.729 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:26 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:26.729 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:26 smithi135 ceph-mon[32689]: from='osd.2 [v2:172.21.15.122:6818/3927725903,v1:172.21.15.122:6819/3927725903]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-11T08:04:26.729 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:26 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:04:26.729 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:26 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:26.729 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:26 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:26.803 DEBUG:teuthology.orchestra.run.smithi122:osd.2> sudo journalctl -f -n 0 -u ceph-f1d75850-7013-11ef-bcea-c7b262605968@osd.2.service 2024-09-11T08:04:26.805 INFO:tasks.cephadm:Deploying osd.3 on smithi122 with /dev/vg_nvme/lv_1... 2024-09-11T08:04:26.805 DEBUG:teuthology.orchestra.run.smithi122:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f1d75850-7013-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-09-11T08:04:27.482 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:04:27.843 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:27 smithi122 ceph-mon[39726]: from='osd.2 [v2:172.21.15.122:6818/3927725903,v1:172.21.15.122:6819/3927725903]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-11T08:04:27.843 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:27 smithi122 ceph-mon[39726]: osdmap e16: 3 total, 2 up, 3 in 2024-09-11T08:04:27.843 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:27 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:04:27.843 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:27 smithi122 ceph-mon[39726]: from='osd.2 [v2:172.21.15.122:6818/3927725903,v1:172.21.15.122:6819/3927725903]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-09-11T08:04:27.843 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:27 smithi122 ceph-mon[39726]: pgmap v69: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:04:27.844 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:27 smithi122 ceph-mon[28992]: from='osd.2 [v2:172.21.15.122:6818/3927725903,v1:172.21.15.122:6819/3927725903]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-11T08:04:27.844 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:27 smithi122 ceph-mon[28992]: osdmap e16: 3 total, 2 up, 3 in 2024-09-11T08:04:27.844 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:27 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:04:27.844 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:27 smithi122 ceph-mon[28992]: from='osd.2 [v2:172.21.15.122:6818/3927725903,v1:172.21.15.122:6819/3927725903]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-09-11T08:04:27.844 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:27 smithi122 ceph-mon[28992]: pgmap v69: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:04:27.844 INFO:journalctl@ceph.osd.2.smithi122.stdout:Sep 11 08:04:27 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-2[55338]: 2024-09-11T08:04:27.483+0000 7efd0ad49640 -1 osd.2 0 waiting for initial osdmap 2024-09-11T08:04:27.844 INFO:journalctl@ceph.osd.2.smithi122.stdout:Sep 11 08:04:27 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-2[55338]: 2024-09-11T08:04:27.492+0000 7efd06381640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T08:04:27.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:27 smithi135 ceph-mon[32689]: from='osd.2 [v2:172.21.15.122:6818/3927725903,v1:172.21.15.122:6819/3927725903]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-11T08:04:27.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:27 smithi135 ceph-mon[32689]: osdmap e16: 3 total, 2 up, 3 in 2024-09-11T08:04:27.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:27 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:04:27.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:27 smithi135 ceph-mon[32689]: from='osd.2 [v2:172.21.15.122:6818/3927725903,v1:172.21.15.122:6819/3927725903]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-09-11T08:04:27.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:27 smithi135 ceph-mon[32689]: pgmap v69: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:04:28.800 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:28 smithi122 ceph-mon[39726]: from='osd.2 [v2:172.21.15.122:6818/3927725903,v1:172.21.15.122:6819/3927725903]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-09-11T08:04:28.801 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:28 smithi122 ceph-mon[39726]: osdmap e17: 3 total, 2 up, 3 in 2024-09-11T08:04:28.801 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:28 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:04:28.801 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:28 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:04:28.801 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:28 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:04:28.801 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:28 smithi122 ceph-mon[28992]: from='osd.2 [v2:172.21.15.122:6818/3927725903,v1:172.21.15.122:6819/3927725903]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-09-11T08:04:28.801 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:28 smithi122 ceph-mon[28992]: osdmap e17: 3 total, 2 up, 3 in 2024-09-11T08:04:28.801 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:28 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:04:28.801 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:28 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:04:28.801 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:28 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:04:28.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:28 smithi135 ceph-mon[32689]: from='osd.2 [v2:172.21.15.122:6818/3927725903,v1:172.21.15.122:6819/3927725903]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-09-11T08:04:28.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:28 smithi135 ceph-mon[32689]: osdmap e17: 3 total, 2 up, 3 in 2024-09-11T08:04:28.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:28 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:04:28.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:28 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:04:28.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:28 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:04:29.675 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:29 smithi122 ceph-mon[39726]: purged_snaps scrub starts 2024-09-11T08:04:29.675 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:29 smithi122 ceph-mon[39726]: purged_snaps scrub ok 2024-09-11T08:04:29.675 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:29 smithi122 ceph-mon[39726]: osd.2 [v2:172.21.15.122:6818/3927725903,v1:172.21.15.122:6819/3927725903] boot 2024-09-11T08:04:29.675 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:29 smithi122 ceph-mon[39726]: osdmap e18: 3 total, 3 up, 3 in 2024-09-11T08:04:29.675 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:29 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:04:29.675 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:29 smithi122 ceph-mon[39726]: pgmap v72: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:04:29.675 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:29 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-09-11T08:04:29.676 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:29 smithi122 ceph-mon[28992]: purged_snaps scrub starts 2024-09-11T08:04:29.676 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:29 smithi122 ceph-mon[28992]: purged_snaps scrub ok 2024-09-11T08:04:29.676 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:29 smithi122 ceph-mon[28992]: osd.2 [v2:172.21.15.122:6818/3927725903,v1:172.21.15.122:6819/3927725903] boot 2024-09-11T08:04:29.676 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:29 smithi122 ceph-mon[28992]: osdmap e18: 3 total, 3 up, 3 in 2024-09-11T08:04:29.676 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:29 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:04:29.676 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:29 smithi122 ceph-mon[28992]: pgmap v72: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:04:29.676 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:29 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-09-11T08:04:29.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:29 smithi135 ceph-mon[32689]: purged_snaps scrub starts 2024-09-11T08:04:29.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:29 smithi135 ceph-mon[32689]: purged_snaps scrub ok 2024-09-11T08:04:29.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:29 smithi135 ceph-mon[32689]: osd.2 [v2:172.21.15.122:6818/3927725903,v1:172.21.15.122:6819/3927725903] boot 2024-09-11T08:04:29.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:29 smithi135 ceph-mon[32689]: osdmap e18: 3 total, 3 up, 3 in 2024-09-11T08:04:29.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:29 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:04:29.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:29 smithi135 ceph-mon[32689]: pgmap v72: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:04:29.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:29 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-09-11T08:04:30.701 INFO:teuthology.orchestra.run.smithi122.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-09-11T08:04:30.702 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:30 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-09-11T08:04:30.702 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:30 smithi122 ceph-mon[28992]: osdmap e19: 3 total, 3 up, 3 in 2024-09-11T08:04:30.702 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:30 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:30.702 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:30 smithi122 ceph-mon[28992]: Detected new or changed devices on smithi122 2024-09-11T08:04:30.702 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:30 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:30.702 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:30 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:30.702 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:30 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:04:30.702 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:30 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:04:30.702 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:30 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:04:30.702 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:30 smithi122 ceph-mon[28992]: Adjusting osd_memory_target on smithi122 to 5354M 2024-09-11T08:04:30.702 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:30 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:30.702 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:30 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:04:30.703 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:30 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:04:30.703 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:30 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:30.703 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:30 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-09-11T08:04:30.703 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:30 smithi122 ceph-mon[39726]: osdmap e19: 3 total, 3 up, 3 in 2024-09-11T08:04:30.703 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:30 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:30.703 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:30 smithi122 ceph-mon[39726]: Detected new or changed devices on smithi122 2024-09-11T08:04:30.703 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:30 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:30.703 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:30 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:30.703 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:30 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:04:30.703 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:30 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:04:30.704 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:30 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:04:30.704 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:30 smithi122 ceph-mon[39726]: Adjusting osd_memory_target on smithi122 to 5354M 2024-09-11T08:04:30.704 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:30 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:30.704 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:30 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:04:30.704 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:30 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:04:30.704 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:30 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:30.800 INFO:teuthology.orchestra.run.smithi122.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-09-11T08:04:30.815 INFO:teuthology.orchestra.run.smithi122.stderr: stderr: 10+0 records in 2024-09-11T08:04:30.815 INFO:teuthology.orchestra.run.smithi122.stderr:10+0 records out 2024-09-11T08:04:30.816 INFO:teuthology.orchestra.run.smithi122.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0130065 s, 806 MB/s 2024-09-11T08:04:30.816 INFO:teuthology.orchestra.run.smithi122.stderr:--> Zapping successful for: 2024-09-11T08:04:30.879 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:30 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-09-11T08:04:30.880 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:30 smithi135 ceph-mon[32689]: osdmap e19: 3 total, 3 up, 3 in 2024-09-11T08:04:30.880 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:30 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:30.880 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:30 smithi135 ceph-mon[32689]: Detected new or changed devices on smithi122 2024-09-11T08:04:30.880 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:30 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:30.880 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:30 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:30.880 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:30 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:04:30.880 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:30 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:04:30.880 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:30 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:04:30.880 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:30 smithi135 ceph-mon[32689]: Adjusting osd_memory_target on smithi122 to 5354M 2024-09-11T08:04:30.880 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:30 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:30.880 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:30 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:04:30.881 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:30 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:04:30.881 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:30 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:31.413 DEBUG:teuthology.orchestra.run.smithi122:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f1d75850-7013-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi122:vg_nvme/lv_1 2024-09-11T08:04:31.533 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:31 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:31.533 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:31 smithi122 ceph-mon[28992]: osdmap e20: 3 total, 3 up, 3 in 2024-09-11T08:04:31.533 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:31 smithi122 ceph-mon[28992]: pgmap v75: 1 pgs: 1 unknown; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:04:31.534 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:31 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:31.534 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:31 smithi122 ceph-mon[39726]: osdmap e20: 3 total, 3 up, 3 in 2024-09-11T08:04:31.534 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:31 smithi122 ceph-mon[39726]: pgmap v75: 1 pgs: 1 unknown; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:04:31.534 INFO:journalctl@ceph.osd.0.smithi122.stdout:Sep 11 08:04:31 smithi122 sudo[58951]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T08:04:31.534 INFO:journalctl@ceph.osd.0.smithi122.stdout:Sep 11 08:04:31 smithi122 sudo[58951]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T08:04:31.534 INFO:journalctl@ceph.osd.0.smithi122.stdout:Sep 11 08:04:31 smithi122 sudo[58951]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T08:04:31.534 INFO:journalctl@ceph.osd.0.smithi122.stdout:Sep 11 08:04:31 smithi122 sudo[58951]: pam_unix(sudo:session): session closed for user root 2024-09-11T08:04:31.673 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:04:31.794 INFO:journalctl@ceph.osd.0.smithi122.stdout:Sep 11 08:04:31 smithi122 sudo[58975]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T08:04:31.794 INFO:journalctl@ceph.osd.0.smithi122.stdout:Sep 11 08:04:31 smithi122 sudo[58975]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T08:04:31.794 INFO:journalctl@ceph.osd.0.smithi122.stdout:Sep 11 08:04:31 smithi122 sudo[58975]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T08:04:31.795 INFO:journalctl@ceph.osd.0.smithi122.stdout:Sep 11 08:04:31 smithi122 sudo[58975]: pam_unix(sudo:session): session closed for user root 2024-09-11T08:04:31.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:31 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:31.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:31 smithi135 ceph-mon[32689]: osdmap e20: 3 total, 3 up, 3 in 2024-09-11T08:04:31.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:31 smithi135 ceph-mon[32689]: pgmap v75: 1 pgs: 1 unknown; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:04:32.067 INFO:journalctl@ceph.osd.1.smithi122.stdout:Sep 11 08:04:31 smithi122 sudo[59031]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T08:04:32.068 INFO:journalctl@ceph.osd.1.smithi122.stdout:Sep 11 08:04:31 smithi122 sudo[59031]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T08:04:32.068 INFO:journalctl@ceph.osd.1.smithi122.stdout:Sep 11 08:04:31 smithi122 sudo[59031]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T08:04:32.068 INFO:journalctl@ceph.osd.1.smithi122.stdout:Sep 11 08:04:31 smithi122 sudo[59031]: pam_unix(sudo:session): session closed for user root 2024-09-11T08:04:32.322 INFO:journalctl@ceph.osd.1.smithi122.stdout:Sep 11 08:04:32 smithi122 sudo[59038]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T08:04:32.322 INFO:journalctl@ceph.osd.1.smithi122.stdout:Sep 11 08:04:32 smithi122 sudo[59038]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T08:04:32.322 INFO:journalctl@ceph.osd.1.smithi122.stdout:Sep 11 08:04:32 smithi122 sudo[59038]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T08:04:32.323 INFO:journalctl@ceph.osd.1.smithi122.stdout:Sep 11 08:04:32 smithi122 sudo[59038]: pam_unix(sudo:session): session closed for user root 2024-09-11T08:04:32.583 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:32 smithi122 ceph-mon[28992]: osdmap e21: 3 total, 3 up, 3 in 2024-09-11T08:04:32.583 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:32 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:32.583 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:32 smithi122 ceph-mon[39726]: osdmap e21: 3 total, 3 up, 3 in 2024-09-11T08:04:32.583 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:32 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:32.583 INFO:journalctl@ceph.osd.2.smithi122.stdout:Sep 11 08:04:32 smithi122 sudo[59054]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T08:04:32.583 INFO:journalctl@ceph.osd.2.smithi122.stdout:Sep 11 08:04:32 smithi122 sudo[59054]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T08:04:32.583 INFO:journalctl@ceph.osd.2.smithi122.stdout:Sep 11 08:04:32 smithi122 sudo[59054]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T08:04:32.584 INFO:journalctl@ceph.osd.2.smithi122.stdout:Sep 11 08:04:32 smithi122 sudo[59054]: pam_unix(sudo:session): session closed for user root 2024-09-11T08:04:32.846 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:32 smithi122 sudo[59066]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T08:04:32.846 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:32 smithi122 sudo[59066]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T08:04:32.846 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:32 smithi122 sudo[59066]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T08:04:32.846 INFO:journalctl@ceph.osd.2.smithi122.stdout:Sep 11 08:04:32 smithi122 sudo[59058]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T08:04:32.847 INFO:journalctl@ceph.osd.2.smithi122.stdout:Sep 11 08:04:32 smithi122 sudo[59058]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T08:04:32.847 INFO:journalctl@ceph.osd.2.smithi122.stdout:Sep 11 08:04:32 smithi122 sudo[59058]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T08:04:32.847 INFO:journalctl@ceph.osd.2.smithi122.stdout:Sep 11 08:04:32 smithi122 sudo[59058]: pam_unix(sudo:session): session closed for user root 2024-09-11T08:04:32.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:32 smithi135 ceph-mon[32689]: osdmap e21: 3 total, 3 up, 3 in 2024-09-11T08:04:32.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:32 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:33.097 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:32 smithi122 sudo[59066]: pam_unix(sudo:session): session closed for user root 2024-09-11T08:04:33.097 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:33 smithi122 sudo[59091]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T08:04:33.097 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:33 smithi122 sudo[59091]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T08:04:33.097 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:33 smithi122 sudo[59091]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T08:04:33.097 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:33 smithi122 sudo[59091]: pam_unix(sudo:session): session closed for user root 2024-09-11T08:04:33.596 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:33 smithi135 sudo[35196]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T08:04:33.596 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:33 smithi135 sudo[35196]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T08:04:33.597 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:33 smithi135 sudo[35196]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T08:04:33.597 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:33 smithi135 sudo[35196]: pam_unix(sudo:session): session closed for user root 2024-09-11T08:04:33.875 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:33 smithi122 ceph-mon[28992]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T08:04:33.875 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:33 smithi122 ceph-mon[28992]: pgmap v77: 1 pgs: 1 active+clean; 320 KiB data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:04:33.875 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:33 smithi122 ceph-mon[28992]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T08:04:33.875 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:33 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:04:33.875 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:33 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:04:33.875 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:33 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:04:33.875 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:33 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:04:33.876 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:33 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:04:33.876 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:33 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:04:33.876 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:33 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:04:33.876 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:33 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:04:33.876 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:33 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:04:33.876 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:33 smithi122 ceph-mon[39726]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T08:04:33.876 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:33 smithi122 ceph-mon[39726]: pgmap v77: 1 pgs: 1 active+clean; 320 KiB data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:04:33.876 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:33 smithi122 ceph-mon[39726]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T08:04:33.876 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:33 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:04:33.876 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:33 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:04:33.876 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:33 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:04:33.877 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:33 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:04:33.877 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:33 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:04:33.877 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:33 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:04:33.877 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:33 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:04:33.877 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:33 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:04:33.877 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:33 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:04:33.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:33 smithi135 sudo[35199]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T08:04:33.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:33 smithi135 sudo[35199]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T08:04:33.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:33 smithi135 sudo[35199]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T08:04:33.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:33 smithi135 sudo[35199]: pam_unix(sudo:session): session closed for user root 2024-09-11T08:04:33.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:33 smithi135 ceph-mon[32689]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T08:04:33.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:33 smithi135 ceph-mon[32689]: pgmap v77: 1 pgs: 1 active+clean; 320 KiB data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:04:33.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:33 smithi135 ceph-mon[32689]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T08:04:33.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:33 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:04:33.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:33 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:04:33.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:33 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:04:33.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:33 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:04:33.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:33 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:04:33.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:33 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:04:33.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:33 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:04:33.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:33 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:04:33.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:33 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:04:34.141 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:33 smithi122 sudo[59236]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T08:04:34.141 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:33 smithi122 sudo[59236]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T08:04:34.141 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:33 smithi122 sudo[59236]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T08:04:34.141 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:33 smithi122 sudo[59236]: pam_unix(sudo:session): session closed for user root 2024-09-11T08:04:34.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:34 smithi122 sudo[59252]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T08:04:34.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:34 smithi122 sudo[59252]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T08:04:34.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:34 smithi122 sudo[59252]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T08:04:34.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:34 smithi122 sudo[59252]: pam_unix(sudo:session): session closed for user root 2024-09-11T08:04:34.890 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:34 smithi122 ceph-mon[28992]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T08:04:34.890 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:34 smithi122 ceph-mon[28992]: from='client.24206 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:04:34.891 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:34 smithi122 ceph-mon[28992]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T08:04:34.891 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:34 smithi122 ceph-mon[28992]: mgrmap e15: y(active, since 2m), standbys: x 2024-09-11T08:04:34.891 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:34 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:34.891 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:34 smithi122 ceph-mon[28992]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T08:04:34.891 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:34 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:04:34.891 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:34 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:04:34.891 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:34 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:04:34.891 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:34 smithi122 ceph-mon[28992]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T08:04:34.891 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:34 smithi122 ceph-mon[39726]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T08:04:34.891 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:34 smithi122 ceph-mon[39726]: from='client.24206 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:04:34.892 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:34 smithi122 ceph-mon[39726]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T08:04:34.892 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:34 smithi122 ceph-mon[39726]: mgrmap e15: y(active, since 2m), standbys: x 2024-09-11T08:04:34.892 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:34 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:34.892 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:34 smithi122 ceph-mon[39726]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T08:04:34.892 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:34 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:04:34.892 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:34 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:04:34.892 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:34 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:04:34.892 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:34 smithi122 ceph-mon[39726]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T08:04:34.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:34 smithi135 ceph-mon[32689]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T08:04:34.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:34 smithi135 ceph-mon[32689]: from='client.24206 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:04:34.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:34 smithi135 ceph-mon[32689]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T08:04:34.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:34 smithi135 ceph-mon[32689]: mgrmap e15: y(active, since 2m), standbys: x 2024-09-11T08:04:34.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:34 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:34.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:34 smithi135 ceph-mon[32689]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T08:04:34.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:34 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:04:34.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:34 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:04:34.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:34 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:04:34.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:34 smithi135 ceph-mon[32689]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T08:04:35.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:35 smithi135 ceph-mon[32689]: pgmap v78: 1 pgs: 1 active+clean; 320 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:04:35.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:35 smithi122 ceph-mon[28992]: pgmap v78: 1 pgs: 1 active+clean; 320 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:04:35.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:35 smithi122 ceph-mon[39726]: pgmap v78: 1 pgs: 1 active+clean; 320 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:04:36.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:36 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/2579679613' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "de322300-e371-42f0-801f-8239a2988813"}]: dispatch 2024-09-11T08:04:36.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:36 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/2579679613' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "de322300-e371-42f0-801f-8239a2988813"}]': finished 2024-09-11T08:04:36.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:36 smithi135 ceph-mon[32689]: osdmap e22: 4 total, 3 up, 4 in 2024-09-11T08:04:36.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:36 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:04:36.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:36 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/3552718298' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:04:36.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:36 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/2579679613' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "de322300-e371-42f0-801f-8239a2988813"}]: dispatch 2024-09-11T08:04:36.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:36 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/2579679613' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "de322300-e371-42f0-801f-8239a2988813"}]': finished 2024-09-11T08:04:36.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:36 smithi122 ceph-mon[39726]: osdmap e22: 4 total, 3 up, 4 in 2024-09-11T08:04:36.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:36 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:04:36.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:36 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/3552718298' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:04:36.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:36 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/2579679613' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "de322300-e371-42f0-801f-8239a2988813"}]: dispatch 2024-09-11T08:04:36.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:36 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/2579679613' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "de322300-e371-42f0-801f-8239a2988813"}]': finished 2024-09-11T08:04:36.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:36 smithi122 ceph-mon[28992]: osdmap e22: 4 total, 3 up, 4 in 2024-09-11T08:04:36.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:36 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:04:36.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:36 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/3552718298' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:04:37.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:37 smithi135 ceph-mon[32689]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 5.2 KiB/s rd, 253 KiB/s wr, 17 op/s 2024-09-11T08:04:37.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:37 smithi122 ceph-mon[28992]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 5.2 KiB/s rd, 253 KiB/s wr, 17 op/s 2024-09-11T08:04:37.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:37 smithi122 ceph-mon[39726]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 5.2 KiB/s rd, 253 KiB/s wr, 17 op/s 2024-09-11T08:04:39.968 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:39 smithi122 ceph-mon[28992]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 875 MiB used, 267 GiB / 268 GiB avail; 3.8 KiB/s rd, 182 KiB/s wr, 12 op/s 2024-09-11T08:04:39.968 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:39 smithi122 ceph-mon[39726]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 875 MiB used, 267 GiB / 268 GiB avail; 3.8 KiB/s rd, 182 KiB/s wr, 12 op/s 2024-09-11T08:04:40.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:39 smithi135 ceph-mon[32689]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 875 MiB used, 267 GiB / 268 GiB avail; 3.8 KiB/s rd, 182 KiB/s wr, 12 op/s 2024-09-11T08:04:41.912 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:41 smithi122 ceph-mon[28992]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.9 KiB/s rd, 143 KiB/s wr, 9 op/s 2024-09-11T08:04:41.913 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:41 smithi122 ceph-mon[39726]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.9 KiB/s rd, 143 KiB/s wr, 9 op/s 2024-09-11T08:04:42.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:41 smithi135 ceph-mon[32689]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.9 KiB/s rd, 143 KiB/s wr, 9 op/s 2024-09-11T08:04:42.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:42 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-11T08:04:42.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:42 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:04:42.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:42 smithi122 ceph-mon[28992]: Deploying daemon osd.3 on smithi122 2024-09-11T08:04:42.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:42 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-11T08:04:42.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:42 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:04:42.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:42 smithi122 ceph-mon[39726]: Deploying daemon osd.3 on smithi122 2024-09-11T08:04:43.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:42 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-11T08:04:43.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:42 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:04:43.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:42 smithi135 ceph-mon[32689]: Deploying daemon osd.3 on smithi122 2024-09-11T08:04:43.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:43 smithi122 ceph-mon[28992]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.7 KiB/s rd, 131 KiB/s wr, 8 op/s 2024-09-11T08:04:43.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:43 smithi122 ceph-mon[39726]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.7 KiB/s rd, 131 KiB/s wr, 8 op/s 2024-09-11T08:04:44.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:43 smithi135 ceph-mon[32689]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.7 KiB/s rd, 131 KiB/s wr, 8 op/s 2024-09-11T08:04:45.999 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:45 smithi122 ceph-mon[39726]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.7 KiB/s rd, 131 KiB/s wr, 8 op/s 2024-09-11T08:04:45.999 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:45 smithi122 ceph-mon[28992]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.7 KiB/s rd, 131 KiB/s wr, 8 op/s 2024-09-11T08:04:46.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:45 smithi135 ceph-mon[32689]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.7 KiB/s rd, 131 KiB/s wr, 8 op/s 2024-09-11T08:04:47.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:47 smithi122 ceph-mon[28992]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.5 KiB/s rd, 120 KiB/s wr, 8 op/s 2024-09-11T08:04:47.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:47 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:04:47.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:47 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:47.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:47 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:47.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:47 smithi122 ceph-mon[39726]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.5 KiB/s rd, 120 KiB/s wr, 8 op/s 2024-09-11T08:04:47.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:47 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:04:47.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:47 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:47.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:47 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:48.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:47 smithi135 ceph-mon[32689]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.5 KiB/s rd, 120 KiB/s wr, 8 op/s 2024-09-11T08:04:48.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:47 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:04:48.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:47 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:48.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:47 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:49.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:49 smithi122 ceph-mon[28992]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:04:49.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:49 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:49.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:49 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:49.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:49 smithi122 ceph-mon[39726]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:04:49.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:49 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:49.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:49 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:50.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:49 smithi135 ceph-mon[32689]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:04:50.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:49 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:50.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:49 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:50.374 INFO:teuthology.orchestra.run.smithi122.stdout:Created osd(s) 3 on host 'smithi122' 2024-09-11T08:04:50.965 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:50 smithi122 ceph-mon[39726]: from='osd.3 [v2:172.21.15.122:6826/3186512798,v1:172.21.15.122:6827/3186512798]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-11T08:04:50.966 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:50 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:04:50.966 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:50 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:04:50.966 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:50 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:50.966 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:50 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:04:50.966 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:50 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:50.966 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:50 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:50.966 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:50 smithi122 ceph-mon[28992]: from='osd.3 [v2:172.21.15.122:6826/3186512798,v1:172.21.15.122:6827/3186512798]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-11T08:04:50.966 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:50 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:04:50.966 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:50 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:04:50.966 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:50 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:50.967 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:50 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:04:50.967 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:50 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:50.967 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:50 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:50.989 DEBUG:teuthology.orchestra.run.smithi122:osd.3> sudo journalctl -f -n 0 -u ceph-f1d75850-7013-11ef-bcea-c7b262605968@osd.3.service 2024-09-11T08:04:50.991 INFO:tasks.cephadm:Deploying osd.4 on smithi135 with /dev/vg_nvme/lv_4... 2024-09-11T08:04:50.991 DEBUG:teuthology.orchestra.run.smithi135:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f1d75850-7013-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-09-11T08:04:51.018 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:50 smithi135 ceph-mon[32689]: from='osd.3 [v2:172.21.15.122:6826/3186512798,v1:172.21.15.122:6827/3186512798]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-11T08:04:51.018 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:50 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:04:51.018 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:50 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:04:51.019 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:50 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:51.019 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:50 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:04:51.019 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:50 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:51.019 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:50 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:51.240 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:04:51.856 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:51 smithi122 ceph-mon[28992]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:04:51.856 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:51 smithi122 ceph-mon[28992]: from='osd.3 [v2:172.21.15.122:6826/3186512798,v1:172.21.15.122:6827/3186512798]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-11T08:04:51.856 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:51 smithi122 ceph-mon[28992]: osdmap e23: 4 total, 3 up, 4 in 2024-09-11T08:04:51.857 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:51 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:04:51.857 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:51 smithi122 ceph-mon[28992]: from='osd.3 [v2:172.21.15.122:6826/3186512798,v1:172.21.15.122:6827/3186512798]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-09-11T08:04:51.857 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:51 smithi122 ceph-mon[39726]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:04:51.857 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:51 smithi122 ceph-mon[39726]: from='osd.3 [v2:172.21.15.122:6826/3186512798,v1:172.21.15.122:6827/3186512798]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-11T08:04:51.857 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:51 smithi122 ceph-mon[39726]: osdmap e23: 4 total, 3 up, 4 in 2024-09-11T08:04:51.857 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:51 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:04:51.857 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:51 smithi122 ceph-mon[39726]: from='osd.3 [v2:172.21.15.122:6826/3186512798,v1:172.21.15.122:6827/3186512798]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-09-11T08:04:51.857 INFO:journalctl@ceph.osd.3.smithi122.stdout:Sep 11 08:04:51 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-3[60981]: 2024-09-11T08:04:51.766+0000 7f15ca493640 -1 osd.3 0 waiting for initial osdmap 2024-09-11T08:04:51.858 INFO:journalctl@ceph.osd.3.smithi122.stdout:Sep 11 08:04:51 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-3[60981]: 2024-09-11T08:04:51.773+0000 7f15c6acd640 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T08:04:51.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:51 smithi135 ceph-mon[32689]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:04:51.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:51 smithi135 ceph-mon[32689]: from='osd.3 [v2:172.21.15.122:6826/3186512798,v1:172.21.15.122:6827/3186512798]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-11T08:04:51.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:51 smithi135 ceph-mon[32689]: osdmap e23: 4 total, 3 up, 4 in 2024-09-11T08:04:51.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:51 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:04:51.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:51 smithi135 ceph-mon[32689]: from='osd.3 [v2:172.21.15.122:6826/3186512798,v1:172.21.15.122:6827/3186512798]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-09-11T08:04:52.898 INFO:teuthology.orchestra.run.smithi135.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-09-11T08:04:52.898 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:52 smithi135 ceph-mon[32689]: from='osd.3 [v2:172.21.15.122:6826/3186512798,v1:172.21.15.122:6827/3186512798]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-09-11T08:04:52.898 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:52 smithi135 ceph-mon[32689]: osdmap e24: 4 total, 3 up, 4 in 2024-09-11T08:04:52.898 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:52 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:04:52.898 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:52 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:04:52.999 INFO:teuthology.orchestra.run.smithi135.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-09-11T08:04:53.015 INFO:teuthology.orchestra.run.smithi135.stderr: stderr: 10+0 records in 2024-09-11T08:04:53.015 INFO:teuthology.orchestra.run.smithi135.stderr:10+0 records out 2024-09-11T08:04:53.015 INFO:teuthology.orchestra.run.smithi135.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0133423 s, 786 MB/s 2024-09-11T08:04:53.015 INFO:teuthology.orchestra.run.smithi135.stderr:--> Zapping successful for: 2024-09-11T08:04:53.116 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:52 smithi122 ceph-mon[28992]: from='osd.3 [v2:172.21.15.122:6826/3186512798,v1:172.21.15.122:6827/3186512798]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-09-11T08:04:53.116 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:52 smithi122 ceph-mon[28992]: osdmap e24: 4 total, 3 up, 4 in 2024-09-11T08:04:53.116 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:52 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:04:53.116 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:52 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:04:53.117 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:52 smithi122 ceph-mon[39726]: from='osd.3 [v2:172.21.15.122:6826/3186512798,v1:172.21.15.122:6827/3186512798]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-09-11T08:04:53.117 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:52 smithi122 ceph-mon[39726]: osdmap e24: 4 total, 3 up, 4 in 2024-09-11T08:04:53.117 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:52 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:04:53.117 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:52 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:04:53.635 DEBUG:teuthology.orchestra.run.smithi135:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f1d75850-7013-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi135:vg_nvme/lv_4 2024-09-11T08:04:53.929 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:04:53.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:53 smithi135 ceph-mon[32689]: purged_snaps scrub starts 2024-09-11T08:04:53.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:53 smithi135 ceph-mon[32689]: purged_snaps scrub ok 2024-09-11T08:04:53.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:53 smithi135 ceph-mon[32689]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:04:53.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:53 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:04:53.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:53 smithi135 ceph-mon[32689]: osd.3 [v2:172.21.15.122:6826/3186512798,v1:172.21.15.122:6827/3186512798] boot 2024-09-11T08:04:53.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:53 smithi135 ceph-mon[32689]: osdmap e25: 4 total, 4 up, 4 in 2024-09-11T08:04:53.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:53 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:04:54.234 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:53 smithi122 ceph-mon[28992]: purged_snaps scrub starts 2024-09-11T08:04:54.234 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:53 smithi122 ceph-mon[28992]: purged_snaps scrub ok 2024-09-11T08:04:54.234 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:53 smithi122 ceph-mon[28992]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:04:54.234 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:53 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:04:54.234 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:53 smithi122 ceph-mon[28992]: osd.3 [v2:172.21.15.122:6826/3186512798,v1:172.21.15.122:6827/3186512798] boot 2024-09-11T08:04:54.234 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:53 smithi122 ceph-mon[28992]: osdmap e25: 4 total, 4 up, 4 in 2024-09-11T08:04:54.235 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:53 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:04:54.235 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:53 smithi122 ceph-mon[39726]: purged_snaps scrub starts 2024-09-11T08:04:54.235 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:53 smithi122 ceph-mon[39726]: purged_snaps scrub ok 2024-09-11T08:04:54.235 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:53 smithi122 ceph-mon[39726]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:04:54.235 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:53 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:04:54.235 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:53 smithi122 ceph-mon[39726]: osd.3 [v2:172.21.15.122:6826/3186512798,v1:172.21.15.122:6827/3186512798] boot 2024-09-11T08:04:54.235 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:53 smithi122 ceph-mon[39726]: osdmap e25: 4 total, 4 up, 4 in 2024-09-11T08:04:54.235 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:53 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:04:55.016 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:54 smithi135 ceph-mon[32689]: osdmap e26: 4 total, 4 up, 4 in 2024-09-11T08:04:55.016 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:54 smithi135 ceph-mon[32689]: Detected new or changed devices on smithi122 2024-09-11T08:04:55.016 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:54 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:55.016 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:54 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:55.016 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:54 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:04:55.016 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:54 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:04:55.017 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:54 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:04:55.017 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:54 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:04:55.017 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:54 smithi135 ceph-mon[32689]: Adjusting osd_memory_target on smithi122 to 4015M 2024-09-11T08:04:55.017 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:54 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:55.017 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:54 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:04:55.017 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:54 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:04:55.017 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:54 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:55.234 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:54 smithi122 ceph-mon[28992]: osdmap e26: 4 total, 4 up, 4 in 2024-09-11T08:04:55.234 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:54 smithi122 ceph-mon[28992]: Detected new or changed devices on smithi122 2024-09-11T08:04:55.234 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:54 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:55.234 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:54 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:55.234 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:54 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:04:55.234 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:54 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:04:55.235 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:54 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:04:55.235 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:54 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:04:55.235 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:54 smithi122 ceph-mon[28992]: Adjusting osd_memory_target on smithi122 to 4015M 2024-09-11T08:04:55.235 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:54 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:55.235 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:54 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:04:55.235 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:54 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:04:55.235 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:54 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:55.235 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:54 smithi122 ceph-mon[39726]: osdmap e26: 4 total, 4 up, 4 in 2024-09-11T08:04:55.235 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:54 smithi122 ceph-mon[39726]: Detected new or changed devices on smithi122 2024-09-11T08:04:55.235 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:54 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:55.235 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:54 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:55.235 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:54 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:04:55.235 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:54 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:04:55.236 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:54 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:04:55.236 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:54 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:04:55.236 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:54 smithi122 ceph-mon[39726]: Adjusting osd_memory_target on smithi122 to 4015M 2024-09-11T08:04:55.236 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:54 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:55.236 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:54 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:04:55.236 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:54 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:04:55.236 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:54 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:04:56.193 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:55 smithi135 ceph-mon[32689]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:04:56.194 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:55 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:04:56.194 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:55 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:04:56.194 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:55 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:04:56.234 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:55 smithi122 ceph-mon[28992]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:04:56.234 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:55 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:04:56.234 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:55 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:04:56.234 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:55 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:04:56.234 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:55 smithi122 ceph-mon[39726]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:04:56.234 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:55 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:04:56.235 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:55 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:04:56.235 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:55 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:04:57.091 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:56 smithi135 ceph-mon[32689]: from='client.24196 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:04:57.234 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:56 smithi122 ceph-mon[39726]: from='client.24196 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:04:57.234 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:56 smithi122 ceph-mon[28992]: from='client.24196 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:04:57.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:57 smithi135 ceph-mon[32689]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:04:58.135 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:57 smithi122 ceph-mon[28992]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:04:58.136 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:57 smithi122 ceph-mon[39726]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:04:59.148 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:58 smithi122 ceph-mon[39726]: from='client.? 172.21.15.135:0/3250032690' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f6ddd756-6005-4794-9ccc-d5d421e1fff6"}]: dispatch 2024-09-11T08:04:59.148 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:58 smithi122 ceph-mon[39726]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f6ddd756-6005-4794-9ccc-d5d421e1fff6"}]: dispatch 2024-09-11T08:04:59.148 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:58 smithi122 ceph-mon[39726]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f6ddd756-6005-4794-9ccc-d5d421e1fff6"}]': finished 2024-09-11T08:04:59.148 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:58 smithi122 ceph-mon[39726]: osdmap e27: 5 total, 4 up, 5 in 2024-09-11T08:04:59.148 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:58 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:04:59.149 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:04:58 smithi122 ceph-mon[39726]: from='client.? 172.21.15.135:0/2507335549' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:04:59.149 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:58 smithi122 ceph-mon[28992]: from='client.? 172.21.15.135:0/3250032690' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f6ddd756-6005-4794-9ccc-d5d421e1fff6"}]: dispatch 2024-09-11T08:04:59.149 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:58 smithi122 ceph-mon[28992]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f6ddd756-6005-4794-9ccc-d5d421e1fff6"}]: dispatch 2024-09-11T08:04:59.149 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:58 smithi122 ceph-mon[28992]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f6ddd756-6005-4794-9ccc-d5d421e1fff6"}]': finished 2024-09-11T08:04:59.149 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:58 smithi122 ceph-mon[28992]: osdmap e27: 5 total, 4 up, 5 in 2024-09-11T08:04:59.149 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:58 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:04:59.149 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:04:58 smithi122 ceph-mon[28992]: from='client.? 172.21.15.135:0/2507335549' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:04:59.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:58 smithi135 ceph-mon[32689]: from='client.? 172.21.15.135:0/3250032690' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f6ddd756-6005-4794-9ccc-d5d421e1fff6"}]: dispatch 2024-09-11T08:04:59.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:58 smithi135 ceph-mon[32689]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f6ddd756-6005-4794-9ccc-d5d421e1fff6"}]: dispatch 2024-09-11T08:04:59.229 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:58 smithi135 ceph-mon[32689]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f6ddd756-6005-4794-9ccc-d5d421e1fff6"}]': finished 2024-09-11T08:04:59.229 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:58 smithi135 ceph-mon[32689]: osdmap e27: 5 total, 4 up, 5 in 2024-09-11T08:04:59.229 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:58 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:04:59.229 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:04:58 smithi135 ceph-mon[32689]: from='client.? 172.21.15.135:0/2507335549' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:05:00.353 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:00 smithi122 ceph-mon[28992]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:05:00.353 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:00 smithi122 ceph-mon[39726]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:05:00.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:00 smithi135 ceph-mon[32689]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:05:02.057 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:02 smithi135 ceph-mon[32689]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:05:02.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:02 smithi122 ceph-mon[28992]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:05:02.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:02 smithi122 ceph-mon[39726]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:05:04.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:04 smithi135 ceph-mon[32689]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:05:04.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:04 smithi122 ceph-mon[28992]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:05:04.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:04 smithi122 ceph-mon[39726]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:05:05.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:05 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-11T08:05:05.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:05 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:05:05.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:05 smithi135 ceph-mon[32689]: Deploying daemon osd.4 on smithi135 2024-09-11T08:05:05.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:05 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-11T08:05:05.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:05 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:05:05.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:05 smithi122 ceph-mon[28992]: Deploying daemon osd.4 on smithi135 2024-09-11T08:05:05.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:05 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-11T08:05:05.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:05 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:05:05.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:05 smithi122 ceph-mon[39726]: Deploying daemon osd.4 on smithi135 2024-09-11T08:05:06.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:06 smithi135 ceph-mon[32689]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:05:06.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:06 smithi122 ceph-mon[28992]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:05:06.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:06 smithi122 ceph-mon[39726]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:05:08.419 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:08 smithi135 ceph-mon[32689]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:05:08.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:08 smithi122 ceph-mon[28992]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:05:08.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:08 smithi122 ceph-mon[39726]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:05:09.413 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:09 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:05:09.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:09 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:05:09.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:09 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:05:10.234 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:10 smithi122 ceph-mon[39726]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:05:10.234 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:10 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:05:10.234 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:10 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:05:10.235 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:10 smithi122 ceph-mon[28992]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:05:10.235 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:10 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:05:10.235 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:10 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:05:10.339 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:10 smithi135 ceph-mon[32689]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:05:10.339 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:10 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:05:10.339 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:10 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:05:12.376 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:12 smithi135 ceph-mon[32689]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:05:12.376 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:12 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:05:12.377 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:12 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:05:12.377 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:12 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:05:12.377 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:12 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:05:12.377 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:12 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:05:12.412 INFO:teuthology.orchestra.run.smithi135.stdout:Created osd(s) 4 on host 'smithi135' 2024-09-11T08:05:12.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:12 smithi122 ceph-mon[28992]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:05:12.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:12 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:05:12.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:12 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:05:12.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:12 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:05:12.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:12 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:05:12.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:12 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:05:12.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:12 smithi122 ceph-mon[39726]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:05:12.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:12 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:05:12.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:12 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:05:12.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:12 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:05:12.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:12 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:05:12.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:12 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:05:13.029 DEBUG:teuthology.orchestra.run.smithi135:osd.4> sudo journalctl -f -n 0 -u ceph-f1d75850-7013-11ef-bcea-c7b262605968@osd.4.service 2024-09-11T08:05:13.032 INFO:tasks.cephadm:Deploying osd.5 on smithi135 with /dev/vg_nvme/lv_3... 2024-09-11T08:05:13.032 DEBUG:teuthology.orchestra.run.smithi135:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f1d75850-7013-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-09-11T08:05:13.257 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:13 smithi135 ceph-mon[32689]: from='osd.4 [v2:172.21.15.135:6800/1495829679,v1:172.21.15.135:6801/1495829679]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T08:05:13.258 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:13 smithi135 ceph-mon[32689]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T08:05:13.258 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:13 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:05:13.258 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:13 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:05:13.258 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:13 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:05:13.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:13 smithi122 ceph-mon[28992]: from='osd.4 [v2:172.21.15.135:6800/1495829679,v1:172.21.15.135:6801/1495829679]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T08:05:13.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:13 smithi122 ceph-mon[28992]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T08:05:13.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:13 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:05:13.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:13 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:05:13.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:13 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:05:13.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:13 smithi122 ceph-mon[39726]: from='osd.4 [v2:172.21.15.135:6800/1495829679,v1:172.21.15.135:6801/1495829679]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T08:05:13.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:13 smithi122 ceph-mon[39726]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T08:05:13.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:13 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:05:13.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:13 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:05:13.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:13 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:05:13.849 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:05:14.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:14 smithi135 ceph-mon[32689]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:05:14.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:14 smithi135 ceph-mon[32689]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-11T08:05:14.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:14 smithi135 ceph-mon[32689]: osdmap e28: 5 total, 4 up, 5 in 2024-09-11T08:05:14.229 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:14 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:05:14.229 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:14 smithi135 ceph-mon[32689]: from='osd.4 [v2:172.21.15.135:6800/1495829679,v1:172.21.15.135:6801/1495829679]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-09-11T08:05:14.229 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:14 smithi135 ceph-mon[32689]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-09-11T08:05:14.229 INFO:journalctl@ceph.osd.4.smithi135.stdout:Sep 11 08:05:14 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-4[37578]: 2024-09-11T08:05:14.046+0000 7fbc24c12640 -1 osd.4 0 waiting for initial osdmap 2024-09-11T08:05:14.229 INFO:journalctl@ceph.osd.4.smithi135.stdout:Sep 11 08:05:14 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-4[37578]: 2024-09-11T08:05:14.050+0000 7fbc1fa27640 -1 osd.4 29 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T08:05:14.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:14 smithi122 ceph-mon[28992]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:05:14.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:14 smithi122 ceph-mon[28992]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-11T08:05:14.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:14 smithi122 ceph-mon[28992]: osdmap e28: 5 total, 4 up, 5 in 2024-09-11T08:05:14.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:14 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:05:14.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:14 smithi122 ceph-mon[28992]: from='osd.4 [v2:172.21.15.135:6800/1495829679,v1:172.21.15.135:6801/1495829679]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-09-11T08:05:14.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:14 smithi122 ceph-mon[28992]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-09-11T08:05:14.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:14 smithi122 ceph-mon[39726]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:05:14.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:14 smithi122 ceph-mon[39726]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-11T08:05:14.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:14 smithi122 ceph-mon[39726]: osdmap e28: 5 total, 4 up, 5 in 2024-09-11T08:05:14.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:14 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:05:14.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:14 smithi122 ceph-mon[39726]: from='osd.4 [v2:172.21.15.135:6800/1495829679,v1:172.21.15.135:6801/1495829679]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-09-11T08:05:14.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:14 smithi122 ceph-mon[39726]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-09-11T08:05:15.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:15 smithi135 ceph-mon[32689]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]': finished 2024-09-11T08:05:15.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:15 smithi135 ceph-mon[32689]: osdmap e29: 5 total, 4 up, 5 in 2024-09-11T08:05:15.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:15 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:05:15.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:15 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:05:15.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:15 smithi135 ceph-mon[32689]: osd.4 [v2:172.21.15.135:6800/1495829679,v1:172.21.15.135:6801/1495829679] boot 2024-09-11T08:05:15.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:15 smithi122 ceph-mon[28992]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]': finished 2024-09-11T08:05:15.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:15 smithi122 ceph-mon[28992]: osdmap e29: 5 total, 4 up, 5 in 2024-09-11T08:05:15.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:15 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:05:15.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:15 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:05:15.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:15 smithi122 ceph-mon[28992]: osd.4 [v2:172.21.15.135:6800/1495829679,v1:172.21.15.135:6801/1495829679] boot 2024-09-11T08:05:15.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:15 smithi122 ceph-mon[39726]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]': finished 2024-09-11T08:05:15.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:15 smithi122 ceph-mon[39726]: osdmap e29: 5 total, 4 up, 5 in 2024-09-11T08:05:15.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:15 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:05:15.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:15 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:05:15.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:15 smithi122 ceph-mon[39726]: osd.4 [v2:172.21.15.135:6800/1495829679,v1:172.21.15.135:6801/1495829679] boot 2024-09-11T08:05:16.147 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:16 smithi135 ceph-mon[32689]: purged_snaps scrub starts 2024-09-11T08:05:16.148 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:16 smithi135 ceph-mon[32689]: purged_snaps scrub ok 2024-09-11T08:05:16.148 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:16 smithi135 ceph-mon[32689]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:05:16.148 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:16 smithi135 ceph-mon[32689]: osdmap e30: 5 total, 5 up, 5 in 2024-09-11T08:05:16.148 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:16 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:05:16.148 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:16 smithi135 ceph-mon[32689]: osdmap e31: 5 total, 5 up, 5 in 2024-09-11T08:05:16.383 INFO:teuthology.orchestra.run.smithi135.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-09-11T08:05:16.480 INFO:teuthology.orchestra.run.smithi135.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-09-11T08:05:16.518 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:16 smithi122 ceph-mon[28992]: purged_snaps scrub starts 2024-09-11T08:05:16.518 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:16 smithi122 ceph-mon[28992]: purged_snaps scrub ok 2024-09-11T08:05:16.518 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:16 smithi122 ceph-mon[28992]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:05:16.518 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:16 smithi122 ceph-mon[28992]: osdmap e30: 5 total, 5 up, 5 in 2024-09-11T08:05:16.519 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:16 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:05:16.519 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:16 smithi122 ceph-mon[28992]: osdmap e31: 5 total, 5 up, 5 in 2024-09-11T08:05:16.519 INFO:teuthology.orchestra.run.smithi135.stderr: stderr: 10+0 records in 2024-09-11T08:05:16.519 INFO:teuthology.orchestra.run.smithi135.stderr:10+0 records out 2024-09-11T08:05:16.519 INFO:teuthology.orchestra.run.smithi135.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0130362 s, 804 MB/s 2024-09-11T08:05:16.519 INFO:teuthology.orchestra.run.smithi135.stderr:--> Zapping successful for: 2024-09-11T08:05:16.519 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:16 smithi122 ceph-mon[39726]: purged_snaps scrub starts 2024-09-11T08:05:16.519 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:16 smithi122 ceph-mon[39726]: purged_snaps scrub ok 2024-09-11T08:05:16.519 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:16 smithi122 ceph-mon[39726]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:05:16.520 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:16 smithi122 ceph-mon[39726]: osdmap e30: 5 total, 5 up, 5 in 2024-09-11T08:05:16.520 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:16 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:05:16.520 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:16 smithi122 ceph-mon[39726]: osdmap e31: 5 total, 5 up, 5 in 2024-09-11T08:05:17.210 DEBUG:teuthology.orchestra.run.smithi135:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f1d75850-7013-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi135:vg_nvme/lv_3 2024-09-11T08:05:17.886 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:05:18.311 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:18 smithi135 ceph-mon[32689]: pgmap v109: 1 pgs: 1 peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:05:18.311 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:18 smithi135 ceph-mon[32689]: osdmap e32: 5 total, 5 up, 5 in 2024-09-11T08:05:18.311 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:18 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:05:18.311 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:18 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:05:18.311 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:18 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:05:18.311 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:18 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:05:18.311 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:18 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:05:18.311 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:18 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:05:18.311 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:18 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:05:18.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:18 smithi122 ceph-mon[39726]: pgmap v109: 1 pgs: 1 peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:05:18.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:18 smithi122 ceph-mon[39726]: osdmap e32: 5 total, 5 up, 5 in 2024-09-11T08:05:18.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:18 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:05:18.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:18 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:05:18.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:18 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:05:18.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:18 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:05:18.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:18 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:05:18.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:18 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:05:18.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:18 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:05:18.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:18 smithi122 ceph-mon[28992]: pgmap v109: 1 pgs: 1 peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:05:18.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:18 smithi122 ceph-mon[28992]: osdmap e32: 5 total, 5 up, 5 in 2024-09-11T08:05:18.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:18 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:05:18.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:18 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:05:18.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:18 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:05:18.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:18 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:05:18.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:18 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:05:18.486 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:18 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:05:18.486 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:18 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:05:19.221 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:19 smithi135 ceph-mon[32689]: Detected new or changed devices on smithi135 2024-09-11T08:05:19.221 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:19 smithi135 ceph-mon[32689]: Adjusting osd_memory_target on smithi135 to 5815M 2024-09-11T08:05:19.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:19 smithi122 ceph-mon[39726]: Detected new or changed devices on smithi135 2024-09-11T08:05:19.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:19 smithi122 ceph-mon[39726]: Adjusting osd_memory_target on smithi135 to 5815M 2024-09-11T08:05:19.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:19 smithi122 ceph-mon[28992]: Detected new or changed devices on smithi135 2024-09-11T08:05:19.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:19 smithi122 ceph-mon[28992]: Adjusting osd_memory_target on smithi135 to 5815M 2024-09-11T08:05:20.306 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:20 smithi135 ceph-mon[32689]: pgmap v111: 1 pgs: 1 peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:05:20.307 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:20 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:05:20.307 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:20 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:05:20.307 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:20 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:05:20.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:20 smithi122 ceph-mon[28992]: pgmap v111: 1 pgs: 1 peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:05:20.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:20 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:05:20.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:20 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:05:20.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:20 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:05:20.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:20 smithi122 ceph-mon[39726]: pgmap v111: 1 pgs: 1 peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:05:20.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:20 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:05:20.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:20 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:05:20.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:20 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:05:21.196 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:21 smithi135 ceph-mon[32689]: from='client.24223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:05:21.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:21 smithi122 ceph-mon[28992]: from='client.24223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:05:21.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:21 smithi122 ceph-mon[39726]: from='client.24223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:05:22.420 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:22 smithi135 ceph-mon[32689]: pgmap v112: 1 pgs: 1 peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:05:22.420 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:22 smithi135 ceph-mon[32689]: from='client.? 172.21.15.135:0/871779581' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a97f08b8-4600-4a5f-8fbb-e55ed79a6f4a"}]: dispatch 2024-09-11T08:05:22.420 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:22 smithi135 ceph-mon[32689]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a97f08b8-4600-4a5f-8fbb-e55ed79a6f4a"}]: dispatch 2024-09-11T08:05:22.420 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:22 smithi135 ceph-mon[32689]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a97f08b8-4600-4a5f-8fbb-e55ed79a6f4a"}]': finished 2024-09-11T08:05:22.420 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:22 smithi135 ceph-mon[32689]: osdmap e33: 6 total, 5 up, 6 in 2024-09-11T08:05:22.421 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:22 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:05:22.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:22 smithi122 ceph-mon[28992]: pgmap v112: 1 pgs: 1 peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:05:22.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:22 smithi122 ceph-mon[28992]: from='client.? 172.21.15.135:0/871779581' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a97f08b8-4600-4a5f-8fbb-e55ed79a6f4a"}]: dispatch 2024-09-11T08:05:22.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:22 smithi122 ceph-mon[28992]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a97f08b8-4600-4a5f-8fbb-e55ed79a6f4a"}]: dispatch 2024-09-11T08:05:22.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:22 smithi122 ceph-mon[28992]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a97f08b8-4600-4a5f-8fbb-e55ed79a6f4a"}]': finished 2024-09-11T08:05:22.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:22 smithi122 ceph-mon[28992]: osdmap e33: 6 total, 5 up, 6 in 2024-09-11T08:05:22.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:22 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:05:22.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:22 smithi122 ceph-mon[39726]: pgmap v112: 1 pgs: 1 peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:05:22.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:22 smithi122 ceph-mon[39726]: from='client.? 172.21.15.135:0/871779581' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a97f08b8-4600-4a5f-8fbb-e55ed79a6f4a"}]: dispatch 2024-09-11T08:05:22.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:22 smithi122 ceph-mon[39726]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a97f08b8-4600-4a5f-8fbb-e55ed79a6f4a"}]: dispatch 2024-09-11T08:05:22.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:22 smithi122 ceph-mon[39726]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a97f08b8-4600-4a5f-8fbb-e55ed79a6f4a"}]': finished 2024-09-11T08:05:22.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:22 smithi122 ceph-mon[39726]: osdmap e33: 6 total, 5 up, 6 in 2024-09-11T08:05:22.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:22 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:05:23.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:23 smithi135 ceph-mon[32689]: from='client.? 172.21.15.135:0/3662552698' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:05:23.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:23 smithi122 ceph-mon[28992]: from='client.? 172.21.15.135:0/3662552698' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:05:23.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:23 smithi122 ceph-mon[39726]: from='client.? 172.21.15.135:0/3662552698' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:05:24.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:24 smithi135 ceph-mon[32689]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-09-11T08:05:24.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:24 smithi122 ceph-mon[28992]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-09-11T08:05:24.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:24 smithi122 ceph-mon[39726]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-09-11T08:05:26.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:26 smithi135 ceph-mon[32689]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T08:05:26.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:26 smithi122 ceph-mon[28992]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T08:05:26.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:26 smithi122 ceph-mon[39726]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T08:05:28.196 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:28 smithi135 ceph-mon[32689]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-11T08:05:28.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:28 smithi122 ceph-mon[28992]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-11T08:05:28.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:28 smithi122 ceph-mon[39726]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-11T08:05:29.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:29 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-11T08:05:29.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:29 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:05:29.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:29 smithi135 ceph-mon[32689]: Deploying daemon osd.5 on smithi135 2024-09-11T08:05:29.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:29 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-11T08:05:29.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:29 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:05:29.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:29 smithi122 ceph-mon[28992]: Deploying daemon osd.5 on smithi135 2024-09-11T08:05:29.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:29 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-11T08:05:29.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:29 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:05:29.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:29 smithi122 ceph-mon[39726]: Deploying daemon osd.5 on smithi135 2024-09-11T08:05:30.150 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:30 smithi135 ceph-mon[32689]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T08:05:30.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:30 smithi122 ceph-mon[28992]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T08:05:30.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:30 smithi122 ceph-mon[39726]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T08:05:32.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:32 smithi135 ceph-mon[32689]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T08:05:32.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:32 smithi122 ceph-mon[28992]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T08:05:32.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:32 smithi122 ceph-mon[39726]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T08:05:34.418 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:34 smithi135 ceph-mon[32689]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:05:34.418 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:34 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:05:34.418 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:34 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:05:34.418 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:34 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:05:34.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:34 smithi122 ceph-mon[28992]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:05:34.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:34 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:05:34.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:34 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:05:34.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:34 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:05:34.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:34 smithi122 ceph-mon[39726]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:05:34.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:34 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:05:34.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:34 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:05:34.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:34 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:05:36.454 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:36 smithi135 ceph-mon[32689]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:05:36.454 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:36 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:05:36.454 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:36 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:05:36.454 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:36 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:05:36.455 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:36 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:05:36.455 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:36 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:05:36.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:36 smithi122 ceph-mon[39726]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:05:36.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:36 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:05:36.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:36 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:05:36.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:36 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:05:36.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:36 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:05:36.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:36 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:05:36.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:36 smithi122 ceph-mon[28992]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:05:36.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:36 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:05:36.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:36 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:05:36.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:36 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:05:36.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:36 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:05:36.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:36 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:05:36.494 INFO:teuthology.orchestra.run.smithi135.stdout:Created osd(s) 5 on host 'smithi135' 2024-09-11T08:05:37.109 DEBUG:teuthology.orchestra.run.smithi135:osd.5> sudo journalctl -f -n 0 -u ceph-f1d75850-7013-11ef-bcea-c7b262605968@osd.5.service 2024-09-11T08:05:37.112 INFO:tasks.cephadm:Deploying osd.6 on smithi135 with /dev/vg_nvme/lv_2... 2024-09-11T08:05:37.112 DEBUG:teuthology.orchestra.run.smithi135:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f1d75850-7013-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-09-11T08:05:37.439 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:37 smithi135 ceph-mon[32689]: from='osd.5 [v2:172.21.15.135:6808/2575765344,v1:172.21.15.135:6809/2575765344]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T08:05:37.440 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:37 smithi135 ceph-mon[32689]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T08:05:37.440 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:37 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:05:37.440 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:37 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:05:37.440 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:37 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:05:37.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:37 smithi122 ceph-mon[28992]: from='osd.5 [v2:172.21.15.135:6808/2575765344,v1:172.21.15.135:6809/2575765344]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T08:05:37.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:37 smithi122 ceph-mon[28992]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T08:05:37.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:37 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:05:37.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:37 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:05:37.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:37 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:05:37.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:37 smithi122 ceph-mon[39726]: from='osd.5 [v2:172.21.15.135:6808/2575765344,v1:172.21.15.135:6809/2575765344]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T08:05:37.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:37 smithi122 ceph-mon[39726]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T08:05:37.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:37 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:05:37.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:37 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:05:37.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:37 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:05:37.861 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:05:38.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:38 smithi135 ceph-mon[32689]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:05:38.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:38 smithi135 ceph-mon[32689]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-11T08:05:38.229 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:38 smithi135 ceph-mon[32689]: osdmap e34: 6 total, 5 up, 6 in 2024-09-11T08:05:38.229 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:38 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:05:38.229 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:38 smithi135 ceph-mon[32689]: from='osd.5 [v2:172.21.15.135:6808/2575765344,v1:172.21.15.135:6809/2575765344]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-09-11T08:05:38.229 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:38 smithi135 ceph-mon[32689]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-09-11T08:05:38.229 INFO:journalctl@ceph.osd.5.smithi135.stdout:Sep 11 08:05:38 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-5[43040]: 2024-09-11T08:05:38.110+0000 7f176b697640 -1 osd.5 0 waiting for initial osdmap 2024-09-11T08:05:38.229 INFO:journalctl@ceph.osd.5.smithi135.stdout:Sep 11 08:05:38 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-5[43040]: 2024-09-11T08:05:38.114+0000 7f17684d2640 -1 osd.5 35 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T08:05:38.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:38 smithi122 ceph-mon[28992]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:05:38.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:38 smithi122 ceph-mon[28992]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-11T08:05:38.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:38 smithi122 ceph-mon[28992]: osdmap e34: 6 total, 5 up, 6 in 2024-09-11T08:05:38.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:38 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:05:38.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:38 smithi122 ceph-mon[28992]: from='osd.5 [v2:172.21.15.135:6808/2575765344,v1:172.21.15.135:6809/2575765344]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-09-11T08:05:38.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:38 smithi122 ceph-mon[28992]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-09-11T08:05:38.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:38 smithi122 ceph-mon[39726]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:05:38.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:38 smithi122 ceph-mon[39726]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-11T08:05:38.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:38 smithi122 ceph-mon[39726]: osdmap e34: 6 total, 5 up, 6 in 2024-09-11T08:05:38.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:38 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:05:38.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:38 smithi122 ceph-mon[39726]: from='osd.5 [v2:172.21.15.135:6808/2575765344,v1:172.21.15.135:6809/2575765344]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-09-11T08:05:38.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:38 smithi122 ceph-mon[39726]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-09-11T08:05:39.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:39 smithi135 ceph-mon[32689]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]': finished 2024-09-11T08:05:39.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:39 smithi135 ceph-mon[32689]: osdmap e35: 6 total, 5 up, 6 in 2024-09-11T08:05:39.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:39 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:05:39.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:39 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:05:39.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:39 smithi122 ceph-mon[28992]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]': finished 2024-09-11T08:05:39.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:39 smithi122 ceph-mon[28992]: osdmap e35: 6 total, 5 up, 6 in 2024-09-11T08:05:39.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:39 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:05:39.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:39 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:05:39.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:39 smithi122 ceph-mon[39726]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]': finished 2024-09-11T08:05:39.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:39 smithi122 ceph-mon[39726]: osdmap e35: 6 total, 5 up, 6 in 2024-09-11T08:05:39.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:39 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:05:39.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:39 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:05:40.150 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:40 smithi135 ceph-mon[32689]: purged_snaps scrub starts 2024-09-11T08:05:40.150 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:40 smithi135 ceph-mon[32689]: purged_snaps scrub ok 2024-09-11T08:05:40.150 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:40 smithi135 ceph-mon[32689]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:05:40.150 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:40 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:05:40.151 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:40 smithi135 ceph-mon[32689]: osd.5 [v2:172.21.15.135:6808/2575765344,v1:172.21.15.135:6809/2575765344] boot 2024-09-11T08:05:40.151 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:40 smithi135 ceph-mon[32689]: osdmap e36: 6 total, 6 up, 6 in 2024-09-11T08:05:40.151 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:40 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:05:40.405 INFO:teuthology.orchestra.run.smithi135.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-09-11T08:05:40.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:40 smithi122 ceph-mon[28992]: purged_snaps scrub starts 2024-09-11T08:05:40.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:40 smithi122 ceph-mon[28992]: purged_snaps scrub ok 2024-09-11T08:05:40.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:40 smithi122 ceph-mon[28992]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:05:40.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:40 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:05:40.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:40 smithi122 ceph-mon[28992]: osd.5 [v2:172.21.15.135:6808/2575765344,v1:172.21.15.135:6809/2575765344] boot 2024-09-11T08:05:40.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:40 smithi122 ceph-mon[28992]: osdmap e36: 6 total, 6 up, 6 in 2024-09-11T08:05:40.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:40 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:05:40.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:40 smithi122 ceph-mon[39726]: purged_snaps scrub starts 2024-09-11T08:05:40.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:40 smithi122 ceph-mon[39726]: purged_snaps scrub ok 2024-09-11T08:05:40.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:40 smithi122 ceph-mon[39726]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:05:40.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:40 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:05:40.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:40 smithi122 ceph-mon[39726]: osd.5 [v2:172.21.15.135:6808/2575765344,v1:172.21.15.135:6809/2575765344] boot 2024-09-11T08:05:40.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:40 smithi122 ceph-mon[39726]: osdmap e36: 6 total, 6 up, 6 in 2024-09-11T08:05:40.486 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:40 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:05:40.506 INFO:teuthology.orchestra.run.smithi135.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-09-11T08:05:40.523 INFO:teuthology.orchestra.run.smithi135.stderr: stderr: 10+0 records in 2024-09-11T08:05:40.523 INFO:teuthology.orchestra.run.smithi135.stderr:10+0 records out 2024-09-11T08:05:40.523 INFO:teuthology.orchestra.run.smithi135.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0140006 s, 749 MB/s 2024-09-11T08:05:40.523 INFO:teuthology.orchestra.run.smithi135.stderr:--> Zapping successful for: 2024-09-11T08:05:41.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:41 smithi135 ceph-mon[32689]: osdmap e37: 6 total, 6 up, 6 in 2024-09-11T08:05:41.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:41 smithi122 ceph-mon[28992]: osdmap e37: 6 total, 6 up, 6 in 2024-09-11T08:05:41.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:41 smithi122 ceph-mon[39726]: osdmap e37: 6 total, 6 up, 6 in 2024-09-11T08:05:41.818 DEBUG:teuthology.orchestra.run.smithi135:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f1d75850-7013-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi135:vg_nvme/lv_2 2024-09-11T08:05:42.062 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:05:42.438 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:42 smithi135 ceph-mon[32689]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:05:42.438 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:42 smithi135 ceph-mon[32689]: osdmap e38: 6 total, 6 up, 6 in 2024-09-11T08:05:42.439 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:42 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:05:42.439 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:42 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:05:42.439 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:42 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:05:42.439 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:42 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:05:42.439 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:42 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:05:42.439 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:42 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:05:42.439 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:42 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:05:42.439 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:42 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:05:42.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:42 smithi122 ceph-mon[28992]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:05:42.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:42 smithi122 ceph-mon[28992]: osdmap e38: 6 total, 6 up, 6 in 2024-09-11T08:05:42.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:42 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:05:42.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:42 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:05:42.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:42 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:05:42.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:42 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:05:42.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:42 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:05:42.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:42 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:05:42.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:42 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:05:42.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:42 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:05:42.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:42 smithi122 ceph-mon[39726]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:05:42.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:42 smithi122 ceph-mon[39726]: osdmap e38: 6 total, 6 up, 6 in 2024-09-11T08:05:42.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:42 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:05:42.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:42 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:05:42.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:42 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:05:42.486 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:42 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:05:42.486 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:42 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:05:42.486 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:42 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:05:42.486 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:42 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:05:42.486 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:42 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:05:43.159 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:43 smithi135 ceph-mon[32689]: Detected new or changed devices on smithi135 2024-09-11T08:05:43.159 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:43 smithi135 ceph-mon[32689]: Adjusting osd_memory_target on smithi135 to 2907M 2024-09-11T08:05:43.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:43 smithi122 ceph-mon[28992]: Detected new or changed devices on smithi135 2024-09-11T08:05:43.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:43 smithi122 ceph-mon[28992]: Adjusting osd_memory_target on smithi135 to 2907M 2024-09-11T08:05:43.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:43 smithi122 ceph-mon[39726]: Detected new or changed devices on smithi135 2024-09-11T08:05:43.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:43 smithi122 ceph-mon[39726]: Adjusting osd_memory_target on smithi135 to 2907M 2024-09-11T08:05:44.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:44 smithi135 ceph-mon[32689]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:05:44.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:44 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:05:44.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:44 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:05:44.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:44 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:05:44.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:44 smithi122 ceph-mon[28992]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:05:44.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:44 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:05:44.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:44 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:05:44.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:44 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:05:44.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:44 smithi122 ceph-mon[39726]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:05:44.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:44 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:05:44.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:44 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:05:44.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:44 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:05:45.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:45 smithi135 ceph-mon[32689]: from='client.24250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:05:45.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:45 smithi122 ceph-mon[28992]: from='client.24250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:05:45.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:45 smithi122 ceph-mon[39726]: from='client.24250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:05:46.398 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:46 smithi135 ceph-mon[32689]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:05:46.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:46 smithi122 ceph-mon[28992]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:05:46.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:46 smithi122 ceph-mon[39726]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:05:47.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:47 smithi135 ceph-mon[32689]: from='client.? 172.21.15.135:0/1275698332' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4caefe69-cb05-4daf-bc91-26097ab2f1f9"}]: dispatch 2024-09-11T08:05:47.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:47 smithi135 ceph-mon[32689]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4caefe69-cb05-4daf-bc91-26097ab2f1f9"}]: dispatch 2024-09-11T08:05:47.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:47 smithi135 ceph-mon[32689]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4caefe69-cb05-4daf-bc91-26097ab2f1f9"}]': finished 2024-09-11T08:05:47.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:47 smithi135 ceph-mon[32689]: osdmap e39: 7 total, 6 up, 7 in 2024-09-11T08:05:47.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:47 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:05:47.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:47 smithi135 ceph-mon[32689]: from='client.? 172.21.15.135:0/2177804616' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:05:47.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:47 smithi122 ceph-mon[28992]: from='client.? 172.21.15.135:0/1275698332' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4caefe69-cb05-4daf-bc91-26097ab2f1f9"}]: dispatch 2024-09-11T08:05:47.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:47 smithi122 ceph-mon[28992]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4caefe69-cb05-4daf-bc91-26097ab2f1f9"}]: dispatch 2024-09-11T08:05:47.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:47 smithi122 ceph-mon[28992]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4caefe69-cb05-4daf-bc91-26097ab2f1f9"}]': finished 2024-09-11T08:05:47.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:47 smithi122 ceph-mon[28992]: osdmap e39: 7 total, 6 up, 7 in 2024-09-11T08:05:47.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:47 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:05:47.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:47 smithi122 ceph-mon[28992]: from='client.? 172.21.15.135:0/2177804616' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:05:47.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:47 smithi122 ceph-mon[39726]: from='client.? 172.21.15.135:0/1275698332' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4caefe69-cb05-4daf-bc91-26097ab2f1f9"}]: dispatch 2024-09-11T08:05:47.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:47 smithi122 ceph-mon[39726]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4caefe69-cb05-4daf-bc91-26097ab2f1f9"}]: dispatch 2024-09-11T08:05:47.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:47 smithi122 ceph-mon[39726]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4caefe69-cb05-4daf-bc91-26097ab2f1f9"}]': finished 2024-09-11T08:05:47.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:47 smithi122 ceph-mon[39726]: osdmap e39: 7 total, 6 up, 7 in 2024-09-11T08:05:47.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:47 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:05:47.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:47 smithi122 ceph-mon[39726]: from='client.? 172.21.15.135:0/2177804616' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:05:48.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:48 smithi135 ceph-mon[32689]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:05:48.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:48 smithi122 ceph-mon[28992]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:05:48.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:48 smithi122 ceph-mon[39726]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:05:50.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:50 smithi135 ceph-mon[32689]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:05:50.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:50 smithi122 ceph-mon[28992]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:05:50.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:50 smithi122 ceph-mon[39726]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:05:52.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:52 smithi135 ceph-mon[32689]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:05:52.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:52 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-11T08:05:52.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:52 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:05:52.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:52 smithi122 ceph-mon[28992]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:05:52.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:52 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-11T08:05:52.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:52 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:05:52.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:52 smithi122 ceph-mon[39726]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:05:52.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:52 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-11T08:05:52.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:52 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:05:53.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:53 smithi135 ceph-mon[32689]: Deploying daemon osd.6 on smithi135 2024-09-11T08:05:53.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:53 smithi122 ceph-mon[28992]: Deploying daemon osd.6 on smithi135 2024-09-11T08:05:53.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:53 smithi122 ceph-mon[39726]: Deploying daemon osd.6 on smithi135 2024-09-11T08:05:54.441 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:54 smithi135 ceph-mon[32689]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:05:54.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:54 smithi122 ceph-mon[28992]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:05:54.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:54 smithi122 ceph-mon[39726]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:05:56.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:56 smithi135 ceph-mon[32689]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:05:56.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:56 smithi122 ceph-mon[28992]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:05:56.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:56 smithi122 ceph-mon[39726]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:05:57.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:57 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:05:57.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:57 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:05:57.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:57 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:05:57.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:57 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:05:57.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:57 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:05:57.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:57 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:05:57.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:57 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:05:57.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:57 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:05:57.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:57 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:05:58.415 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:05:58 smithi135 ceph-mon[32689]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:05:58.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:05:58 smithi122 ceph-mon[39726]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:05:58.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:05:58 smithi122 ceph-mon[28992]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:06:00.137 INFO:teuthology.orchestra.run.smithi135.stdout:Created osd(s) 6 on host 'smithi135' 2024-09-11T08:06:00.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:00 smithi135 ceph-mon[32689]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:06:00.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:00 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:00.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:00 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:00.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:00 smithi122 ceph-mon[28992]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:06:00.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:00 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:00.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:00 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:00.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:00 smithi122 ceph-mon[39726]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:06:00.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:00 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:00.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:00 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:01.129 DEBUG:teuthology.orchestra.run.smithi135:osd.6> sudo journalctl -f -n 0 -u ceph-f1d75850-7013-11ef-bcea-c7b262605968@osd.6.service 2024-09-11T08:06:01.132 INFO:tasks.cephadm:Deploying osd.7 on smithi135 with /dev/vg_nvme/lv_1... 2024-09-11T08:06:01.132 DEBUG:teuthology.orchestra.run.smithi135:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f1d75850-7013-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-09-11T08:06:01.426 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:06:01.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:01 smithi135 ceph-mon[32689]: from='osd.6 [v2:172.21.15.135:6816/1417277044,v1:172.21.15.135:6817/1417277044]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-11T08:06:01.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:01 smithi135 ceph-mon[32689]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-11T08:06:01.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:01 smithi122 ceph-mon[28992]: from='osd.6 [v2:172.21.15.135:6816/1417277044,v1:172.21.15.135:6817/1417277044]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-11T08:06:01.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:01 smithi122 ceph-mon[28992]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-11T08:06:01.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:01 smithi122 ceph-mon[39726]: from='osd.6 [v2:172.21.15.135:6816/1417277044,v1:172.21.15.135:6817/1417277044]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-11T08:06:01.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:01 smithi122 ceph-mon[39726]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-11T08:06:02.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:02 smithi135 ceph-mon[32689]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:06:02.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:02 smithi135 ceph-mon[32689]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-11T08:06:02.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:02 smithi135 ceph-mon[32689]: osdmap e40: 7 total, 6 up, 7 in 2024-09-11T08:06:02.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:02 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:06:02.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:02 smithi135 ceph-mon[32689]: from='osd.6 [v2:172.21.15.135:6816/1417277044,v1:172.21.15.135:6817/1417277044]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-09-11T08:06:02.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:02 smithi135 ceph-mon[32689]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-09-11T08:06:02.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:02 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:02.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:02 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:02.479 INFO:journalctl@ceph.osd.6.smithi135.stdout:Sep 11 08:06:02 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-6[48960]: 2024-09-11T08:06:02.223+0000 7fa59a83d640 -1 osd.6 0 waiting for initial osdmap 2024-09-11T08:06:02.479 INFO:journalctl@ceph.osd.6.smithi135.stdout:Sep 11 08:06:02 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-6[48960]: 2024-09-11T08:06:02.230+0000 7fa593e4f640 -1 osd.6 41 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T08:06:02.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:02 smithi122 ceph-mon[28992]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:06:02.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:02 smithi122 ceph-mon[28992]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-11T08:06:02.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:02 smithi122 ceph-mon[28992]: osdmap e40: 7 total, 6 up, 7 in 2024-09-11T08:06:02.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:02 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:06:02.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:02 smithi122 ceph-mon[28992]: from='osd.6 [v2:172.21.15.135:6816/1417277044,v1:172.21.15.135:6817/1417277044]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-09-11T08:06:02.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:02 smithi122 ceph-mon[28992]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-09-11T08:06:02.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:02 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:02.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:02 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:02.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:02 smithi122 ceph-mon[39726]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:06:02.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:02 smithi122 ceph-mon[39726]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-11T08:06:02.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:02 smithi122 ceph-mon[39726]: osdmap e40: 7 total, 6 up, 7 in 2024-09-11T08:06:02.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:02 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:06:02.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:02 smithi122 ceph-mon[39726]: from='osd.6 [v2:172.21.15.135:6816/1417277044,v1:172.21.15.135:6817/1417277044]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-09-11T08:06:02.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:02 smithi122 ceph-mon[39726]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-09-11T08:06:02.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:02 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:02.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:02 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:03.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:03 smithi135 ceph-mon[32689]: purged_snaps scrub starts 2024-09-11T08:06:03.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:03 smithi135 ceph-mon[32689]: purged_snaps scrub ok 2024-09-11T08:06:03.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:03 smithi135 ceph-mon[32689]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]': finished 2024-09-11T08:06:03.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:03 smithi135 ceph-mon[32689]: osdmap e41: 7 total, 6 up, 7 in 2024-09-11T08:06:03.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:03 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:06:03.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:03 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:06:03.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:03 smithi135 ceph-mon[32689]: osd.6 [v2:172.21.15.135:6816/1417277044,v1:172.21.15.135:6817/1417277044] boot 2024-09-11T08:06:03.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:03 smithi135 ceph-mon[32689]: osdmap e42: 7 total, 7 up, 7 in 2024-09-11T08:06:03.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:03 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:06:03.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:03 smithi122 ceph-mon[39726]: purged_snaps scrub starts 2024-09-11T08:06:03.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:03 smithi122 ceph-mon[39726]: purged_snaps scrub ok 2024-09-11T08:06:03.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:03 smithi122 ceph-mon[39726]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]': finished 2024-09-11T08:06:03.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:03 smithi122 ceph-mon[39726]: osdmap e41: 7 total, 6 up, 7 in 2024-09-11T08:06:03.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:03 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:06:03.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:03 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:06:03.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:03 smithi122 ceph-mon[39726]: osd.6 [v2:172.21.15.135:6816/1417277044,v1:172.21.15.135:6817/1417277044] boot 2024-09-11T08:06:03.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:03 smithi122 ceph-mon[39726]: osdmap e42: 7 total, 7 up, 7 in 2024-09-11T08:06:03.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:03 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:06:03.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:03 smithi122 ceph-mon[28992]: purged_snaps scrub starts 2024-09-11T08:06:03.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:03 smithi122 ceph-mon[28992]: purged_snaps scrub ok 2024-09-11T08:06:03.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:03 smithi122 ceph-mon[28992]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]': finished 2024-09-11T08:06:03.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:03 smithi122 ceph-mon[28992]: osdmap e41: 7 total, 6 up, 7 in 2024-09-11T08:06:03.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:03 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:06:03.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:03 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:06:03.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:03 smithi122 ceph-mon[28992]: osd.6 [v2:172.21.15.135:6816/1417277044,v1:172.21.15.135:6817/1417277044] boot 2024-09-11T08:06:03.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:03 smithi122 ceph-mon[28992]: osdmap e42: 7 total, 7 up, 7 in 2024-09-11T08:06:03.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:03 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:06:04.175 INFO:teuthology.orchestra.run.smithi135.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-09-11T08:06:04.269 INFO:teuthology.orchestra.run.smithi135.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-09-11T08:06:04.289 INFO:teuthology.orchestra.run.smithi135.stderr: stderr: 10+0 records in 2024-09-11T08:06:04.290 INFO:teuthology.orchestra.run.smithi135.stderr:10+0 records out 2024-09-11T08:06:04.290 INFO:teuthology.orchestra.run.smithi135.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0142511 s, 736 MB/s 2024-09-11T08:06:04.290 INFO:teuthology.orchestra.run.smithi135.stderr:--> Zapping successful for: 2024-09-11T08:06:04.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:04 smithi135 ceph-mon[32689]: pgmap v142: 1 pgs: 1 peering; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:06:04.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:04 smithi135 ceph-mon[32689]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-11T08:06:04.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:04 smithi135 ceph-mon[32689]: osdmap e43: 7 total, 7 up, 7 in 2024-09-11T08:06:04.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:04 smithi122 ceph-mon[28992]: pgmap v142: 1 pgs: 1 peering; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:06:04.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:04 smithi122 ceph-mon[28992]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-11T08:06:04.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:04 smithi122 ceph-mon[28992]: osdmap e43: 7 total, 7 up, 7 in 2024-09-11T08:06:04.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:04 smithi122 ceph-mon[39726]: pgmap v142: 1 pgs: 1 peering; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:06:04.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:04 smithi122 ceph-mon[39726]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-11T08:06:04.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:04 smithi122 ceph-mon[39726]: osdmap e43: 7 total, 7 up, 7 in 2024-09-11T08:06:05.182 DEBUG:teuthology.orchestra.run.smithi135:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f1d75850-7013-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi135:vg_nvme/lv_1 2024-09-11T08:06:05.942 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:06:06.390 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:06 smithi135 ceph-mon[32689]: pgmap v145: 1 pgs: 1 peering; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:06:06.390 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:06 smithi135 ceph-mon[32689]: osdmap e44: 7 total, 7 up, 7 in 2024-09-11T08:06:06.391 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:06 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:06.391 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:06 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:06.391 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:06 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:06:06.391 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:06 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:06:06.391 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:06 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:06:06.391 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:06 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:06.391 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:06 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:06:06.391 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:06 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:06:06.391 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:06 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:06.391 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:06 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:06:06.391 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:06 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:06:06.392 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:06 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:06:06.392 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:06 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:06.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:06 smithi122 ceph-mon[28992]: pgmap v145: 1 pgs: 1 peering; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:06:06.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:06 smithi122 ceph-mon[28992]: osdmap e44: 7 total, 7 up, 7 in 2024-09-11T08:06:06.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:06 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:06.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:06 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:06.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:06 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:06:06.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:06 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:06:06.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:06 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:06:06.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:06 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:06.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:06 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:06:06.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:06 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:06:06.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:06 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:06.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:06 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:06:06.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:06 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:06:06.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:06 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:06:06.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:06 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:06.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:06 smithi122 ceph-mon[39726]: pgmap v145: 1 pgs: 1 peering; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:06:06.486 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:06 smithi122 ceph-mon[39726]: osdmap e44: 7 total, 7 up, 7 in 2024-09-11T08:06:06.486 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:06 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:06.486 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:06 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:06.486 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:06 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:06:06.486 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:06 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:06:06.486 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:06 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:06:06.486 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:06 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:06.486 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:06 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:06:06.486 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:06 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:06:06.486 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:06 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:06.486 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:06 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:06:06.486 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:06 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:06:06.486 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:06 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:06:06.487 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:06 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:07.374 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:07 smithi135 ceph-mon[32689]: Detected new or changed devices on smithi135 2024-09-11T08:06:07.375 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:07 smithi135 ceph-mon[32689]: Adjusting osd_memory_target on smithi135 to 1938M 2024-09-11T08:06:07.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:07 smithi122 ceph-mon[28992]: Detected new or changed devices on smithi135 2024-09-11T08:06:07.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:07 smithi122 ceph-mon[28992]: Adjusting osd_memory_target on smithi135 to 1938M 2024-09-11T08:06:07.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:07 smithi122 ceph-mon[39726]: Detected new or changed devices on smithi135 2024-09-11T08:06:07.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:07 smithi122 ceph-mon[39726]: Adjusting osd_memory_target on smithi135 to 1938M 2024-09-11T08:06:08.452 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:08 smithi135 ceph-mon[32689]: pgmap v147: 1 pgs: 1 peering; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:06:08.453 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:08 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:06:08.453 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:08 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:06:08.453 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:08 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:06:08.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:08 smithi122 ceph-mon[28992]: pgmap v147: 1 pgs: 1 peering; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:06:08.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:08 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:06:08.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:08 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:06:08.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:08 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:06:08.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:08 smithi122 ceph-mon[39726]: pgmap v147: 1 pgs: 1 peering; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:06:08.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:08 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:06:08.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:08 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:06:08.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:08 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:06:09.323 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:09 smithi135 ceph-mon[32689]: from='client.24277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:06:09.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:09 smithi122 ceph-mon[28992]: from='client.24277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:06:09.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:09 smithi122 ceph-mon[39726]: from='client.24277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:06:10.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:10 smithi135 ceph-mon[32689]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-11T08:06:10.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:10 smithi135 ceph-mon[32689]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-11T08:06:10.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:10 smithi135 ceph-mon[32689]: Cluster is now healthy 2024-09-11T08:06:10.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:10 smithi135 ceph-mon[32689]: from='client.? 172.21.15.135:0/1777248430' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "be9c6707-0b7d-444c-b6a2-00a69d926260"}]: dispatch 2024-09-11T08:06:10.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:10 smithi135 ceph-mon[32689]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "be9c6707-0b7d-444c-b6a2-00a69d926260"}]: dispatch 2024-09-11T08:06:10.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:10 smithi135 ceph-mon[32689]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "be9c6707-0b7d-444c-b6a2-00a69d926260"}]': finished 2024-09-11T08:06:10.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:10 smithi135 ceph-mon[32689]: osdmap e45: 8 total, 7 up, 8 in 2024-09-11T08:06:10.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:10 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:06:10.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:10 smithi122 ceph-mon[28992]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-11T08:06:10.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:10 smithi122 ceph-mon[28992]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-11T08:06:10.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:10 smithi122 ceph-mon[28992]: Cluster is now healthy 2024-09-11T08:06:10.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:10 smithi122 ceph-mon[28992]: from='client.? 172.21.15.135:0/1777248430' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "be9c6707-0b7d-444c-b6a2-00a69d926260"}]: dispatch 2024-09-11T08:06:10.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:10 smithi122 ceph-mon[28992]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "be9c6707-0b7d-444c-b6a2-00a69d926260"}]: dispatch 2024-09-11T08:06:10.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:10 smithi122 ceph-mon[28992]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "be9c6707-0b7d-444c-b6a2-00a69d926260"}]': finished 2024-09-11T08:06:10.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:10 smithi122 ceph-mon[28992]: osdmap e45: 8 total, 7 up, 8 in 2024-09-11T08:06:10.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:10 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:06:10.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:10 smithi122 ceph-mon[39726]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-11T08:06:10.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:10 smithi122 ceph-mon[39726]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-11T08:06:10.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:10 smithi122 ceph-mon[39726]: Cluster is now healthy 2024-09-11T08:06:10.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:10 smithi122 ceph-mon[39726]: from='client.? 172.21.15.135:0/1777248430' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "be9c6707-0b7d-444c-b6a2-00a69d926260"}]: dispatch 2024-09-11T08:06:10.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:10 smithi122 ceph-mon[39726]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "be9c6707-0b7d-444c-b6a2-00a69d926260"}]: dispatch 2024-09-11T08:06:10.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:10 smithi122 ceph-mon[39726]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "be9c6707-0b7d-444c-b6a2-00a69d926260"}]': finished 2024-09-11T08:06:10.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:10 smithi122 ceph-mon[39726]: osdmap e45: 8 total, 7 up, 8 in 2024-09-11T08:06:10.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:10 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:06:11.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:11 smithi135 ceph-mon[32689]: from='client.? 172.21.15.135:0/2836011450' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:06:11.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:11 smithi122 ceph-mon[28992]: from='client.? 172.21.15.135:0/2836011450' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:06:11.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:11 smithi122 ceph-mon[39726]: from='client.? 172.21.15.135:0/2836011450' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:06:12.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:12 smithi135 ceph-mon[32689]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-09-11T08:06:12.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:12 smithi122 ceph-mon[28992]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-09-11T08:06:12.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:12 smithi122 ceph-mon[39726]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-09-11T08:06:14.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:14 smithi135 ceph-mon[32689]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T08:06:14.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:14 smithi122 ceph-mon[28992]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T08:06:14.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:14 smithi122 ceph-mon[39726]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T08:06:16.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:16 smithi135 ceph-mon[32689]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-11T08:06:16.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:16 smithi122 ceph-mon[39726]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-11T08:06:16.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:16 smithi122 ceph-mon[28992]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-11T08:06:17.376 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:17 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-11T08:06:17.376 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:17 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:06:17.376 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:17 smithi135 ceph-mon[32689]: Deploying daemon osd.7 on smithi135 2024-09-11T08:06:17.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:17 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-11T08:06:17.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:17 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:06:17.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:17 smithi122 ceph-mon[39726]: Deploying daemon osd.7 on smithi135 2024-09-11T08:06:17.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:17 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-11T08:06:17.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:17 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:06:17.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:17 smithi122 ceph-mon[28992]: Deploying daemon osd.7 on smithi135 2024-09-11T08:06:18.185 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:18 smithi135 ceph-mon[32689]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T08:06:18.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:18 smithi122 ceph-mon[28992]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T08:06:18.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:18 smithi122 ceph-mon[39726]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T08:06:20.461 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:20 smithi135 ceph-mon[32689]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:06:20.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:20 smithi122 ceph-mon[28992]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:06:20.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:20 smithi122 ceph-mon[39726]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:06:22.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:22 smithi135 ceph-mon[32689]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:06:22.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:22 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:06:22.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:22 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:22.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:22 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:22.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:22 smithi122 ceph-mon[28992]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:06:22.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:22 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:06:22.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:22 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:22.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:22 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:22.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:22 smithi122 ceph-mon[39726]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:06:22.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:22 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:06:22.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:22 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:22.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:22 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:24.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:24 smithi135 ceph-mon[32689]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:06:24.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:24 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:24.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:24 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:24.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:24 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:06:24.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:24 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:06:24.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:24 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:24.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:24 smithi122 ceph-mon[28992]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:06:24.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:24 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:24.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:24 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:24.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:24 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:06:24.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:24 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:06:24.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:24 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:24.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:24 smithi122 ceph-mon[39726]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:06:24.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:24 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:24.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:24 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:24.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:24 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:06:24.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:24 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:06:24.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:24 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:24.778 INFO:teuthology.orchestra.run.smithi135.stdout:Created osd(s) 7 on host 'smithi135' 2024-09-11T08:06:25.369 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:25 smithi135 ceph-mon[32689]: from='osd.7 [v2:172.21.15.135:6824/1015573664,v1:172.21.15.135:6825/1015573664]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T08:06:25.369 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:25 smithi135 ceph-mon[32689]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T08:06:25.369 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:25 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:06:25.369 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:25 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:25.369 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:25 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:25.394 DEBUG:teuthology.orchestra.run.smithi135:osd.7> sudo journalctl -f -n 0 -u ceph-f1d75850-7013-11ef-bcea-c7b262605968@osd.7.service 2024-09-11T08:06:25.397 INFO:tasks.cephadm:Waiting for 8 OSDs to come up... 2024-09-11T08:06:25.397 DEBUG:teuthology.orchestra.run.smithi122:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f1d75850-7013-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-11T08:06:25.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:25 smithi122 ceph-mon[28992]: from='osd.7 [v2:172.21.15.135:6824/1015573664,v1:172.21.15.135:6825/1015573664]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T08:06:25.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:25 smithi122 ceph-mon[28992]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T08:06:25.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:25 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:06:25.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:25 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:25.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:25 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:25.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:25 smithi122 ceph-mon[39726]: from='osd.7 [v2:172.21.15.135:6824/1015573664,v1:172.21.15.135:6825/1015573664]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T08:06:25.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:25 smithi122 ceph-mon[39726]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T08:06:25.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:25 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:06:25.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:25 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:25.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:25 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:25.650 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:06:26.477 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:26 smithi135 ceph-mon[32689]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:06:26.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:26 smithi135 ceph-mon[32689]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-11T08:06:26.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:26 smithi135 ceph-mon[32689]: osdmap e46: 8 total, 7 up, 8 in 2024-09-11T08:06:26.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:26 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:06:26.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:26 smithi135 ceph-mon[32689]: from='osd.7 [v2:172.21.15.135:6824/1015573664,v1:172.21.15.135:6825/1015573664]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-09-11T08:06:26.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:26 smithi135 ceph-mon[32689]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-09-11T08:06:26.478 INFO:journalctl@ceph.osd.7.smithi135.stdout:Sep 11 08:06:26 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-7[54744]: 2024-09-11T08:06:26.188+0000 7faee1230640 -1 osd.7 0 waiting for initial osdmap 2024-09-11T08:06:26.478 INFO:journalctl@ceph.osd.7.smithi135.stdout:Sep 11 08:06:26 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-7[54744]: 2024-09-11T08:06:26.195+0000 7faeda842640 -1 osd.7 47 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T08:06:26.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:26 smithi122 ceph-mon[39726]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:06:26.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:26 smithi122 ceph-mon[39726]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-11T08:06:26.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:26 smithi122 ceph-mon[39726]: osdmap e46: 8 total, 7 up, 8 in 2024-09-11T08:06:26.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:26 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:06:26.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:26 smithi122 ceph-mon[39726]: from='osd.7 [v2:172.21.15.135:6824/1015573664,v1:172.21.15.135:6825/1015573664]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-09-11T08:06:26.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:26 smithi122 ceph-mon[39726]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-09-11T08:06:26.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:26 smithi122 ceph-mon[28992]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:06:26.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:26 smithi122 ceph-mon[28992]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-11T08:06:26.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:26 smithi122 ceph-mon[28992]: osdmap e46: 8 total, 7 up, 8 in 2024-09-11T08:06:26.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:26 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:06:26.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:26 smithi122 ceph-mon[28992]: from='osd.7 [v2:172.21.15.135:6824/1015573664,v1:172.21.15.135:6825/1015573664]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-09-11T08:06:26.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:26 smithi122 ceph-mon[28992]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-09-11T08:06:27.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:27 smithi135 ceph-mon[32689]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]': finished 2024-09-11T08:06:27.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:27 smithi135 ceph-mon[32689]: osdmap e47: 8 total, 7 up, 8 in 2024-09-11T08:06:27.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:27 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:06:27.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:27 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:06:27.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:27 smithi122 ceph-mon[28992]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]': finished 2024-09-11T08:06:27.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:27 smithi122 ceph-mon[28992]: osdmap e47: 8 total, 7 up, 8 in 2024-09-11T08:06:27.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:27 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:06:27.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:27 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:06:27.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:27 smithi122 ceph-mon[39726]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]': finished 2024-09-11T08:06:27.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:27 smithi122 ceph-mon[39726]: osdmap e47: 8 total, 7 up, 8 in 2024-09-11T08:06:27.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:27 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:06:27.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:27 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:06:27.498 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T08:06:28.254 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":48,"num_osds":8,"num_up_osds":8,"osd_up_since":1726041987,"num_in_osds":8,"osd_in_since":1726041969,"num_remapped_pgs":1} 2024-09-11T08:06:28.255 DEBUG:teuthology.orchestra.run.smithi122:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid f1d75850-7013-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T08:06:28.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:28 smithi135 ceph-mon[32689]: purged_snaps scrub starts 2024-09-11T08:06:28.480 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:28 smithi135 ceph-mon[32689]: purged_snaps scrub ok 2024-09-11T08:06:28.480 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:28 smithi135 ceph-mon[32689]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:06:28.480 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:28 smithi135 ceph-mon[32689]: osd.7 [v2:172.21.15.135:6824/1015573664,v1:172.21.15.135:6825/1015573664] boot 2024-09-11T08:06:28.480 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:28 smithi135 ceph-mon[32689]: osdmap e48: 8 total, 8 up, 8 in 2024-09-11T08:06:28.480 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:28 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:06:28.480 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:28 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/1316568560' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T08:06:28.480 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:28 smithi135 ceph-mon[32689]: osdmap e49: 8 total, 8 up, 8 in 2024-09-11T08:06:28.480 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:28 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:28.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:28 smithi122 ceph-mon[39726]: purged_snaps scrub starts 2024-09-11T08:06:28.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:28 smithi122 ceph-mon[39726]: purged_snaps scrub ok 2024-09-11T08:06:28.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:28 smithi122 ceph-mon[39726]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:06:28.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:28 smithi122 ceph-mon[39726]: osd.7 [v2:172.21.15.135:6824/1015573664,v1:172.21.15.135:6825/1015573664] boot 2024-09-11T08:06:28.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:28 smithi122 ceph-mon[39726]: osdmap e48: 8 total, 8 up, 8 in 2024-09-11T08:06:28.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:28 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:06:28.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:28 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/1316568560' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T08:06:28.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:28 smithi122 ceph-mon[39726]: osdmap e49: 8 total, 8 up, 8 in 2024-09-11T08:06:28.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:28 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:28.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:28 smithi122 ceph-mon[28992]: purged_snaps scrub starts 2024-09-11T08:06:28.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:28 smithi122 ceph-mon[28992]: purged_snaps scrub ok 2024-09-11T08:06:28.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:28 smithi122 ceph-mon[28992]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:06:28.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:28 smithi122 ceph-mon[28992]: osd.7 [v2:172.21.15.135:6824/1015573664,v1:172.21.15.135:6825/1015573664] boot 2024-09-11T08:06:28.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:28 smithi122 ceph-mon[28992]: osdmap e48: 8 total, 8 up, 8 in 2024-09-11T08:06:28.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:28 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:06:28.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:28 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/1316568560' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T08:06:28.486 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:28 smithi122 ceph-mon[28992]: osdmap e49: 8 total, 8 up, 8 in 2024-09-11T08:06:28.486 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:28 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:28.504 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:06:29.204 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:29 smithi122 ceph-mon[28992]: Detected new or changed devices on smithi135 2024-09-11T08:06:29.204 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:29 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:29.204 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:29 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:06:29.205 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:29 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:06:29.205 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:29 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:06:29.205 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:29 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:06:29.205 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:29 smithi122 ceph-mon[28992]: Adjusting osd_memory_target on smithi135 to 1453M 2024-09-11T08:06:29.205 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:29 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:29.205 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:29 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:06:29.205 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:29 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:06:29.205 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:29 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:29.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:29 smithi135 ceph-mon[32689]: Detected new or changed devices on smithi135 2024-09-11T08:06:29.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:29 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:29.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:29 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:06:29.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:29 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:06:29.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:29 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:06:29.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:29 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:06:29.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:29 smithi135 ceph-mon[32689]: Adjusting osd_memory_target on smithi135 to 1453M 2024-09-11T08:06:29.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:29 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:29.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:29 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:06:29.480 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:29 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:06:29.480 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:29 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:29.480 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:29 smithi135 ceph-mon[32689]: osdmap e50: 8 total, 8 up, 8 in 2024-09-11T08:06:29.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:29 smithi122 ceph-mon[28992]: osdmap e50: 8 total, 8 up, 8 in 2024-09-11T08:06:29.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:29 smithi122 ceph-mon[39726]: Detected new or changed devices on smithi135 2024-09-11T08:06:29.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:29 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:29.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:29 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:06:29.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:29 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:06:29.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:29 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:06:29.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:29 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:06:29.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:29 smithi122 ceph-mon[39726]: Adjusting osd_memory_target on smithi135 to 1453M 2024-09-11T08:06:29.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:29 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:29.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:29 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:06:29.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:29 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:06:29.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:29 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:29.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:29 smithi122 ceph-mon[39726]: osdmap e50: 8 total, 8 up, 8 in 2024-09-11T08:06:30.412 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T08:06:30.412 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":50,"fsid":"f1d75850-7013-11ef-bcea-c7b262605968","created":"2024-09-11T08:01:33.023519+0000","modified":"2024-09-11T08:06:29.121495+0000","last_up_change":"2024-09-11T08:06:27.186494+0000","last_in_change":"2024-09-11T08:06:09.969108+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T08:04:28.972292+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"17a779cc-5769-4022-a66e-e4ef151dd3ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":578396583},{"type":"v1","addr":"172.21.15.122:6803","nonce":578396583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":578396583},{"type":"v1","addr":"172.21.15.122:6805","nonce":578396583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":578396583},{"type":"v1","addr":"172.21.15.122:6809","nonce":578396583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":578396583},{"type":"v1","addr":"172.21.15.122:6807","nonce":578396583}]},"public_addr":"172.21.15.122:6803/578396583","cluster_addr":"172.21.15.122:6805/578396583","heartbeat_back_addr":"172.21.15.122:6809/578396583","heartbeat_front_addr":"172.21.15.122:6807/578396583","state":["exists","up"]},{"osd":1,"uuid":"7b89946c-b41a-4c21-ba99-69cf6c474d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":3517945576},{"type":"v1","addr":"172.21.15.122:6811","nonce":3517945576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":3517945576},{"type":"v1","addr":"172.21.15.122:6813","nonce":3517945576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":3517945576},{"type":"v1","addr":"172.21.15.122:6817","nonce":3517945576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":3517945576},{"type":"v1","addr":"172.21.15.122:6815","nonce":3517945576}]},"public_addr":"172.21.15.122:6811/3517945576","cluster_addr":"172.21.15.122:6813/3517945576","heartbeat_back_addr":"172.21.15.122:6817/3517945576","heartbeat_front_addr":"172.21.15.122:6815/3517945576","state":["exists","up"]},{"osd":2,"uuid":"aa9a050b-2829-4235-b379-60d327384fa1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":3927725903},{"type":"v1","addr":"172.21.15.122:6819","nonce":3927725903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":3927725903},{"type":"v1","addr":"172.21.15.122:6821","nonce":3927725903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":3927725903},{"type":"v1","addr":"172.21.15.122:6825","nonce":3927725903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":3927725903},{"type":"v1","addr":"172.21.15.122:6823","nonce":3927725903}]},"public_addr":"172.21.15.122:6819/3927725903","cluster_addr":"172.21.15.122:6821/3927725903","heartbeat_back_addr":"172.21.15.122:6825/3927725903","heartbeat_front_addr":"172.21.15.122:6823/3927725903","state":["exists","up"]},{"osd":3,"uuid":"de322300-e371-42f0-801f-8239a2988813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":3186512798},{"type":"v1","addr":"172.21.15.122:6827","nonce":3186512798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":3186512798},{"type":"v1","addr":"172.21.15.122:6829","nonce":3186512798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3186512798},{"type":"v1","addr":"172.21.15.122:6833","nonce":3186512798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":3186512798},{"type":"v1","addr":"172.21.15.122:6831","nonce":3186512798}]},"public_addr":"172.21.15.122:6827/3186512798","cluster_addr":"172.21.15.122:6829/3186512798","heartbeat_back_addr":"172.21.15.122:6833/3186512798","heartbeat_front_addr":"172.21.15.122:6831/3186512798","state":["exists","up"]},{"osd":4,"uuid":"f6ddd756-6005-4794-9ccc-d5d421e1fff6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1495829679},{"type":"v1","addr":"172.21.15.135:6801","nonce":1495829679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1495829679},{"type":"v1","addr":"172.21.15.135:6803","nonce":1495829679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1495829679},{"type":"v1","addr":"172.21.15.135:6807","nonce":1495829679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1495829679},{"type":"v1","addr":"172.21.15.135:6805","nonce":1495829679}]},"public_addr":"172.21.15.135:6801/1495829679","cluster_addr":"172.21.15.135:6803/1495829679","heartbeat_back_addr":"172.21.15.135:6807/1495829679","heartbeat_front_addr":"172.21.15.135:6805/1495829679","state":["exists","up"]},{"osd":5,"uuid":"a97f08b8-4600-4a5f-8fbb-e55ed79a6f4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":2575765344},{"type":"v1","addr":"172.21.15.135:6809","nonce":2575765344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":2575765344},{"type":"v1","addr":"172.21.15.135:6811","nonce":2575765344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":2575765344},{"type":"v1","addr":"172.21.15.135:6815","nonce":2575765344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":2575765344},{"type":"v1","addr":"172.21.15.135:6813","nonce":2575765344}]},"public_addr":"172.21.15.135:6809/2575765344","cluster_addr":"172.21.15.135:6811/2575765344","heartbeat_back_addr":"172.21.15.135:6815/2575765344","heartbeat_front_addr":"172.21.15.135:6813/2575765344","state":["exists","up"]},{"osd":6,"uuid":"4caefe69-cb05-4daf-bc91-26097ab2f1f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":1417277044},{"type":"v1","addr":"172.21.15.135:6817","nonce":1417277044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":1417277044},{"type":"v1","addr":"172.21.15.135:6819","nonce":1417277044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":1417277044},{"type":"v1","addr":"172.21.15.135:6823","nonce":1417277044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":1417277044},{"type":"v1","addr":"172.21.15.135:6821","nonce":1417277044}]},"public_addr":"172.21.15.135:6817/1417277044","cluster_addr":"172.21.15.135:6819/1417277044","heartbeat_back_addr":"172.21.15.135:6823/1417277044","heartbeat_front_addr":"172.21.15.135:6821/1417277044","state":["exists","up"]},{"osd":7,"uuid":"be9c6707-0b7d-444c-b6a2-00a69d926260","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":1015573664},{"type":"v1","addr":"172.21.15.135:6825","nonce":1015573664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":1015573664},{"type":"v1","addr":"172.21.15.135:6827","nonce":1015573664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":1015573664},{"type":"v1","addr":"172.21.15.135:6831","nonce":1015573664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":1015573664},{"type":"v1","addr":"172.21.15.135:6829","nonce":1015573664}]},"public_addr":"172.21.15.135:6825/1015573664","cluster_addr":"172.21.15.135:6827/1015573664","heartbeat_back_addr":"172.21.15.135:6831/1015573664","heartbeat_front_addr":"172.21.15.135:6829/1015573664","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:03:39.962238+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:04:03.628854+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:04:26.828010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:04:50.984767+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:05:13.149808+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:05:37.236520+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:06:01.213253+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:06:25.436767+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.122:0/2755262646":"2024-09-12T08:02:20.655827+0000","172.21.15.122:0/1663411035":"2024-09-12T08:02:20.655827+0000","172.21.15.122:6801/1221292339":"2024-09-12T08:02:20.655827+0000","172.21.15.122:6800/1221292339":"2024-09-12T08:02:20.655827+0000","172.21.15.122:0/1815778193":"2024-09-12T08:02:20.655827+0000","172.21.15.122:0/3783853267":"2024-09-12T08:01:58.413985+0000","172.21.15.122:0/199541964":"2024-09-12T08:01:58.413985+0000","172.21.15.122:6800/4195848883":"2024-09-12T08:01:58.413985+0000","172.21.15.122:6801/4195848883":"2024-09-12T08:01:58.413985+0000","172.21.15.122:0/1932082800":"2024-09-12T08:01:58.413985+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T08:06:30.413 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:30 smithi122 ceph-mon[28992]: pgmap v163: 1 pgs: 1 peering; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:06:30.413 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:30 smithi122 ceph-mon[39726]: pgmap v163: 1 pgs: 1 peering; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:06:30.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:30 smithi135 ceph-mon[32689]: pgmap v163: 1 pgs: 1 peering; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:06:31.032 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-11T08:04:28.972292+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2024-09-11T08:06:31.032 DEBUG:teuthology.orchestra.run.smithi122:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid f1d75850-7013-11ef-bcea-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-11T08:06:31.273 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:06:31.394 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:31 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/1629080746' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T08:06:31.394 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:31 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/1629080746' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T08:06:31.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:31 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/1629080746' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T08:06:32.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:32 smithi135 ceph-mon[32689]: pgmap v165: 1 pgs: 1 peering; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:06:32.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:32 smithi122 ceph-mon[28992]: pgmap v165: 1 pgs: 1 peering; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:06:32.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:32 smithi122 ceph-mon[39726]: pgmap v165: 1 pgs: 1 peering; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:06:33.124 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 1 2024-09-11T08:06:33.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:33 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/3178737298' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-11T08:06:33.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:33 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/3178737298' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-11T08:06:33.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:33 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/3178737298' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-11T08:06:33.814 INFO:tasks.cephadm:Adding ceph.rgw.foo.a on smithi122 2024-09-11T08:06:33.815 DEBUG:teuthology.orchestra.run.smithi135:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f1d75850-7013-11ef-bcea-c7b262605968 -- ceph orch apply rgw foo.a --placement '1;smithi122=foo.a' 2024-09-11T08:06:34.072 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:06:34.466 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:34 smithi135 ceph-mon[32689]: pgmap v166: 1 pgs: 1 peering; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:06:34.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:34 smithi122 ceph-mon[39726]: pgmap v166: 1 pgs: 1 peering; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:06:34.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:34 smithi122 ceph-mon[28992]: pgmap v166: 1 pgs: 1 peering; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:06:35.727 INFO:teuthology.orchestra.run.smithi135.stdout:Scheduled rgw.foo.a update... 2024-09-11T08:06:36.352 DEBUG:teuthology.orchestra.run.smithi122:rgw.foo.a> sudo journalctl -f -n 0 -u ceph-f1d75850-7013-11ef-bcea-c7b262605968@rgw.foo.a.service 2024-09-11T08:06:36.355 INFO:tasks.cephadm:Adding ceph.iscsi.iscsi.a on smithi135 2024-09-11T08:06:36.355 DEBUG:teuthology.orchestra.run.smithi135:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f1d75850-7013-11ef-bcea-c7b262605968 -- ceph osd pool create datapool 3 3 replicated 2024-09-11T08:06:36.476 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:36 smithi122 ceph-mon[39726]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:06:36.476 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:36 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:36.477 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:36 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:06:36.477 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:36 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:06:36.477 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:36 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:06:36.477 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:36 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:36.477 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:36 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-09-11T08:06:36.477 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:36 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-11T08:06:36.477 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:36 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:36.477 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:36 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:06:36.477 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:36 smithi122 ceph-mon[28992]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:06:36.477 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:36 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:36.478 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:36 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:06:36.478 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:36 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:06:36.478 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:36 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:06:36.478 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:36 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:36.478 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:36 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-09-11T08:06:36.478 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:36 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-11T08:06:36.478 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:36 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:36.478 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:36 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:06:36.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:36 smithi135 ceph-mon[32689]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:06:36.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:36 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:36.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:36 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:06:36.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:36 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:06:36.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:36 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:06:36.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:36 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:36.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:36 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-09-11T08:06:36.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:36 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-11T08:06:36.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:36 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:36.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:36 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:06:36.599 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:06:37.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:37 smithi122 ceph-mon[28992]: from='client.24316 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi122=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:06:37.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:37 smithi122 ceph-mon[28992]: Saving service rgw.foo.a spec with placement smithi122=foo.a;count:1 2024-09-11T08:06:37.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:37 smithi122 ceph-mon[28992]: Deploying daemon rgw.foo.a on smithi122 2024-09-11T08:06:37.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:37 smithi122 ceph-mon[39726]: from='client.24316 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi122=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:06:37.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:37 smithi122 ceph-mon[39726]: Saving service rgw.foo.a spec with placement smithi122=foo.a;count:1 2024-09-11T08:06:37.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:37 smithi122 ceph-mon[39726]: Deploying daemon rgw.foo.a on smithi122 2024-09-11T08:06:37.637 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:37 smithi135 ceph-mon[32689]: from='client.24316 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi122=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:06:37.637 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:37 smithi135 ceph-mon[32689]: Saving service rgw.foo.a spec with placement smithi122=foo.a;count:1 2024-09-11T08:06:37.638 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:37 smithi135 ceph-mon[32689]: Deploying daemon rgw.foo.a on smithi122 2024-09-11T08:06:37.878 INFO:journalctl@ceph.rgw.foo.a.smithi122.stdout:Sep 11 08:06:37 smithi122 systemd[1]: Starting Ceph rgw.foo.a for f1d75850-7013-11ef-bcea-c7b262605968... 2024-09-11T08:06:38.151 INFO:journalctl@ceph.rgw.foo.a.smithi122.stdout:Sep 11 08:06:37 smithi122 podman[65592]: 2024-09-11 08:06:37.778963845 +0000 UTC m=+0.015872183 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T08:06:38.151 INFO:journalctl@ceph.rgw.foo.a.smithi122.stdout:Sep 11 08:06:37 smithi122 podman[65592]: 2024-09-11 08:06:37.881397366 +0000 UTC m=+0.118305694 container create a6a48059e8e9b2c7474e4034debb10ac18ad7d73a6e857ad49da141ea9f1468a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-rgw-foo-a, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-11T08:06:38.151 INFO:journalctl@ceph.rgw.foo.a.smithi122.stdout:Sep 11 08:06:38 smithi122 podman[65592]: 2024-09-11 08:06:38.075507674 +0000 UTC m=+0.312416002 container init a6a48059e8e9b2c7474e4034debb10ac18ad7d73a6e857ad49da141ea9f1468a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-rgw-foo-a, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-11T08:06:38.151 INFO:journalctl@ceph.rgw.foo.a.smithi122.stdout:Sep 11 08:06:38 smithi122 podman[65592]: 2024-09-11 08:06:38.078421523 +0000 UTC m=+0.315329854 container start a6a48059e8e9b2c7474e4034debb10ac18ad7d73a6e857ad49da141ea9f1468a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-rgw-foo-a, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-11T08:06:38.152 INFO:journalctl@ceph.rgw.foo.a.smithi122.stdout:Sep 11 08:06:38 smithi122 bash[65592]: a6a48059e8e9b2c7474e4034debb10ac18ad7d73a6e857ad49da141ea9f1468a 2024-09-11T08:06:38.484 INFO:journalctl@ceph.rgw.foo.a.smithi122.stdout:Sep 11 08:06:38 smithi122 systemd[1]: Started Ceph rgw.foo.a for f1d75850-7013-11ef-bcea-c7b262605968. 2024-09-11T08:06:38.594 INFO:teuthology.orchestra.run.smithi135.stderr:pool 'datapool' created 2024-09-11T08:06:38.894 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:38 smithi135 ceph-mon[32689]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:06:38.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:38 smithi122 ceph-mon[28992]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:06:38.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:38 smithi122 ceph-mon[39726]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:06:39.228 DEBUG:teuthology.orchestra.run.smithi135:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f1d75850-7013-11ef-bcea-c7b262605968 -- rbd pool init datapool 2024-09-11T08:06:39.469 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:06:39.835 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:39 smithi135 ceph-mon[32689]: from='client.? 172.21.15.135:0/3544882142' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-11T08:06:39.835 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:39 smithi135 ceph-mon[32689]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-11T08:06:39.835 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:39 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:39.835 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:39 smithi135 ceph-mon[32689]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-11T08:06:39.835 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:39 smithi135 ceph-mon[32689]: osdmap e51: 8 total, 8 up, 8 in 2024-09-11T08:06:39.835 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:39 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/1567332866' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-11T08:06:39.835 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:39 smithi135 ceph-mon[32689]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-11T08:06:39.835 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:39 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:39.836 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:39 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:39.836 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:39 smithi135 ceph-mon[32689]: Saving service rgw.foo.a spec with placement smithi122=foo.a;count:1 2024-09-11T08:06:39.836 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:39 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:39.836 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:39 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:39.836 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:39 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:06:39.836 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:39 smithi135 ceph-mon[32689]: pgmap v170: 36 pgs: 35 unknown, 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:06:39.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:39 smithi122 ceph-mon[28992]: from='client.? 172.21.15.135:0/3544882142' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-11T08:06:39.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:39 smithi122 ceph-mon[28992]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-11T08:06:39.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:39 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:39.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:39 smithi122 ceph-mon[28992]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-11T08:06:39.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:39 smithi122 ceph-mon[28992]: osdmap e51: 8 total, 8 up, 8 in 2024-09-11T08:06:39.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:39 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/1567332866' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-11T08:06:39.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:39 smithi122 ceph-mon[28992]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-11T08:06:39.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:39 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:39.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:39 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:39.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:39 smithi122 ceph-mon[28992]: Saving service rgw.foo.a spec with placement smithi122=foo.a;count:1 2024-09-11T08:06:39.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:39 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:39.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:39 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:39.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:39 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:06:39.986 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:39 smithi122 ceph-mon[28992]: pgmap v170: 36 pgs: 35 unknown, 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:06:39.986 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:39 smithi122 ceph-mon[39726]: from='client.? 172.21.15.135:0/3544882142' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-11T08:06:39.986 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:39 smithi122 ceph-mon[39726]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-11T08:06:39.986 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:39 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:39.986 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:39 smithi122 ceph-mon[39726]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-11T08:06:39.986 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:39 smithi122 ceph-mon[39726]: osdmap e51: 8 total, 8 up, 8 in 2024-09-11T08:06:39.986 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:39 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/1567332866' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-11T08:06:39.986 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:39 smithi122 ceph-mon[39726]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-11T08:06:39.986 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:39 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:39.986 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:39 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:39.986 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:39 smithi122 ceph-mon[39726]: Saving service rgw.foo.a spec with placement smithi122=foo.a;count:1 2024-09-11T08:06:39.987 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:39 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:39.987 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:39 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:39.987 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:39 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:06:39.987 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:39 smithi122 ceph-mon[39726]: pgmap v170: 36 pgs: 35 unknown, 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:06:40.744 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:40 smithi135 ceph-mon[32689]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-11T08:06:40.745 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:40 smithi135 ceph-mon[32689]: osdmap e52: 8 total, 8 up, 8 in 2024-09-11T08:06:40.745 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:40 smithi135 ceph-mon[32689]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:06:40.745 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:40 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:40.745 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:40 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:40.745 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:40 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:06:40.745 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:40 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:06:40.745 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:40 smithi135 ceph-mon[32689]: Checking dashboard <-> RGW credentials 2024-09-11T08:06:40.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:40 smithi122 ceph-mon[28992]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-11T08:06:40.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:40 smithi122 ceph-mon[28992]: osdmap e52: 8 total, 8 up, 8 in 2024-09-11T08:06:40.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:40 smithi122 ceph-mon[28992]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:06:40.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:40 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:40.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:40 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:40.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:40 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:06:40.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:40 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:06:40.986 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:40 smithi122 ceph-mon[28992]: Checking dashboard <-> RGW credentials 2024-09-11T08:06:40.986 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:40 smithi122 ceph-mon[39726]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-11T08:06:40.986 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:40 smithi122 ceph-mon[39726]: osdmap e52: 8 total, 8 up, 8 in 2024-09-11T08:06:40.986 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:40 smithi122 ceph-mon[39726]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:06:40.987 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:40 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:40.987 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:40 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:40.987 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:40 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:06:40.987 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:40 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:06:40.987 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:40 smithi122 ceph-mon[39726]: Checking dashboard <-> RGW credentials 2024-09-11T08:06:41.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:41 smithi135 ceph-mon[32689]: osdmap e53: 8 total, 8 up, 8 in 2024-09-11T08:06:41.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:41 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/2697947160' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-11T08:06:41.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:41 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/1567332866' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-11T08:06:41.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:41 smithi135 ceph-mon[32689]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-11T08:06:41.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:41 smithi135 ceph-mon[32689]: pgmap v173: 68 pgs: 8 creating+peering, 53 unknown, 7 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-11T08:06:41.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:41 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:41.980 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:41 smithi135 ceph-mon[32689]: from='client.? 172.21.15.135:0/1811590041' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-11T08:06:41.980 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:41 smithi135 ceph-mon[32689]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-11T08:06:41.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:41 smithi122 ceph-mon[28992]: osdmap e53: 8 total, 8 up, 8 in 2024-09-11T08:06:41.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:41 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/2697947160' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-11T08:06:41.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:41 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/1567332866' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-11T08:06:41.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:41 smithi122 ceph-mon[28992]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-11T08:06:41.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:41 smithi122 ceph-mon[28992]: pgmap v173: 68 pgs: 8 creating+peering, 53 unknown, 7 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-11T08:06:41.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:41 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:41.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:41 smithi122 ceph-mon[28992]: from='client.? 172.21.15.135:0/1811590041' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-11T08:06:41.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:41 smithi122 ceph-mon[28992]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-11T08:06:41.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:41 smithi122 ceph-mon[39726]: osdmap e53: 8 total, 8 up, 8 in 2024-09-11T08:06:41.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:41 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/2697947160' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-11T08:06:41.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:41 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/1567332866' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-11T08:06:41.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:41 smithi122 ceph-mon[39726]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-11T08:06:41.986 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:41 smithi122 ceph-mon[39726]: pgmap v173: 68 pgs: 8 creating+peering, 53 unknown, 7 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-11T08:06:41.986 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:41 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:41.986 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:41 smithi122 ceph-mon[39726]: from='client.? 172.21.15.135:0/1811590041' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-11T08:06:41.986 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:41 smithi122 ceph-mon[39726]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-11T08:06:42.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:42 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/2697947160' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-11T08:06:42.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:42 smithi135 ceph-mon[32689]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-11T08:06:42.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:42 smithi135 ceph-mon[32689]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-11T08:06:42.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:42 smithi135 ceph-mon[32689]: osdmap e54: 8 total, 8 up, 8 in 2024-09-11T08:06:42.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:42 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/2697947160' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-11T08:06:42.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:42 smithi122 ceph-mon[28992]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-11T08:06:42.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:42 smithi122 ceph-mon[28992]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-11T08:06:42.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:42 smithi122 ceph-mon[28992]: osdmap e54: 8 total, 8 up, 8 in 2024-09-11T08:06:42.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:42 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/2697947160' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-11T08:06:42.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:42 smithi122 ceph-mon[39726]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-11T08:06:42.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:42 smithi122 ceph-mon[39726]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-11T08:06:42.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:42 smithi122 ceph-mon[39726]: osdmap e54: 8 total, 8 up, 8 in 2024-09-11T08:06:43.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:43 smithi135 ceph-mon[32689]: osdmap e55: 8 total, 8 up, 8 in 2024-09-11T08:06:43.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:43 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/1567332866' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-11T08:06:43.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:43 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/2697947160' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-11T08:06:43.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:43 smithi135 ceph-mon[32689]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-11T08:06:43.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:43 smithi135 ceph-mon[32689]: pgmap v176: 100 pgs: 3 creating+activating, 13 creating+peering, 65 unknown, 19 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-11T08:06:43.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:43 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/2697947160' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-11T08:06:43.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:43 smithi135 ceph-mon[32689]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-11T08:06:43.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:43 smithi135 ceph-mon[32689]: osdmap e56: 8 total, 8 up, 8 in 2024-09-11T08:06:43.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:43 smithi122 ceph-mon[28992]: osdmap e55: 8 total, 8 up, 8 in 2024-09-11T08:06:43.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:43 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/1567332866' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-11T08:06:43.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:43 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/2697947160' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-11T08:06:43.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:43 smithi122 ceph-mon[28992]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-11T08:06:43.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:43 smithi122 ceph-mon[28992]: pgmap v176: 100 pgs: 3 creating+activating, 13 creating+peering, 65 unknown, 19 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-11T08:06:43.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:43 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/2697947160' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-11T08:06:43.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:43 smithi122 ceph-mon[28992]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-11T08:06:43.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:43 smithi122 ceph-mon[28992]: osdmap e56: 8 total, 8 up, 8 in 2024-09-11T08:06:43.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:43 smithi122 ceph-mon[39726]: osdmap e55: 8 total, 8 up, 8 in 2024-09-11T08:06:43.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:43 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/1567332866' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-11T08:06:43.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:43 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/2697947160' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-11T08:06:43.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:43 smithi122 ceph-mon[39726]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-11T08:06:43.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:43 smithi122 ceph-mon[39726]: pgmap v176: 100 pgs: 3 creating+activating, 13 creating+peering, 65 unknown, 19 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-11T08:06:43.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:43 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/2697947160' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-11T08:06:43.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:43 smithi122 ceph-mon[39726]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-11T08:06:43.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:43 smithi122 ceph-mon[39726]: osdmap e56: 8 total, 8 up, 8 in 2024-09-11T08:06:44.287 DEBUG:teuthology.orchestra.run.smithi135:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f1d75850-7013-11ef-bcea-c7b262605968 -- ceph orch apply iscsi datapool admin admin --trusted_ip_list 172.21.15.135 --placement '1;smithi135=iscsi.a' 2024-09-11T08:06:44.531 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:06:45.790 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:45 smithi135 ceph-mon[32689]: osdmap e57: 8 total, 8 up, 8 in 2024-09-11T08:06:45.791 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:45 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/1005381998' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T08:06:45.791 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:45 smithi135 ceph-mon[32689]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T08:06:45.791 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:45 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/3983672372' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T08:06:45.791 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:45 smithi135 ceph-mon[32689]: pgmap v179: 132 pgs: 3 creating+activating, 12 creating+peering, 59 unknown, 58 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 767 B/s wr, 1 op/s 2024-09-11T08:06:45.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:45 smithi122 ceph-mon[39726]: osdmap e57: 8 total, 8 up, 8 in 2024-09-11T08:06:45.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:45 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/1005381998' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T08:06:45.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:45 smithi122 ceph-mon[39726]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T08:06:45.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:45 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/3983672372' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T08:06:45.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:45 smithi122 ceph-mon[39726]: pgmap v179: 132 pgs: 3 creating+activating, 12 creating+peering, 59 unknown, 58 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 767 B/s wr, 1 op/s 2024-09-11T08:06:45.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:45 smithi122 ceph-mon[28992]: osdmap e57: 8 total, 8 up, 8 in 2024-09-11T08:06:45.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:45 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/1005381998' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T08:06:45.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:45 smithi122 ceph-mon[28992]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T08:06:45.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:45 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/3983672372' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T08:06:45.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:45 smithi122 ceph-mon[28992]: pgmap v179: 132 pgs: 3 creating+activating, 12 creating+peering, 59 unknown, 58 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 767 B/s wr, 1 op/s 2024-09-11T08:06:46.158 INFO:teuthology.orchestra.run.smithi135.stdout:Scheduled iscsi.datapool update... 2024-09-11T08:06:46.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:46 smithi135 ceph-mon[32689]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-11T08:06:46.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:46 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/3983672372' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-11T08:06:46.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:46 smithi135 ceph-mon[32689]: osdmap e58: 8 total, 8 up, 8 in 2024-09-11T08:06:46.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:46 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/1005381998' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T08:06:46.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:46 smithi135 ceph-mon[32689]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T08:06:46.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:46 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/3983672372' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T08:06:46.729 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:46 smithi135 ceph-mon[32689]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.135", "placement": "1;smithi135=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:06:46.729 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:46 smithi135 ceph-mon[32689]: Saving service iscsi.datapool spec with placement smithi135=iscsi.a;count:1 2024-09-11T08:06:46.729 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:46 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:46.837 INFO:tasks.cephadm:Distributing iscsi-gateway.cfg... 2024-09-11T08:06:46.838 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-11T08:06:46.838 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2024-09-11T08:06:46.867 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-09-11T08:06:46.868 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2024-09-11T08:06:46.896 DEBUG:teuthology.orchestra.run.smithi135:iscsi.iscsi.a> sudo journalctl -f -n 0 -u ceph-f1d75850-7013-11ef-bcea-c7b262605968@iscsi.iscsi.a.service 2024-09-11T08:06:46.939 INFO:tasks.cephadm:Adding prometheus.a on smithi135 2024-09-11T08:06:46.939 DEBUG:teuthology.orchestra.run.smithi135:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f1d75850-7013-11ef-bcea-c7b262605968 -- ceph orch apply prometheus '1;smithi135=a' 2024-09-11T08:06:46.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:46 smithi122 ceph-mon[28992]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-11T08:06:46.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:46 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/3983672372' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-11T08:06:46.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:46 smithi122 ceph-mon[28992]: osdmap e58: 8 total, 8 up, 8 in 2024-09-11T08:06:46.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:46 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/1005381998' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T08:06:46.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:46 smithi122 ceph-mon[28992]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T08:06:46.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:46 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/3983672372' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T08:06:46.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:46 smithi122 ceph-mon[28992]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.135", "placement": "1;smithi135=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:06:46.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:46 smithi122 ceph-mon[28992]: Saving service iscsi.datapool spec with placement smithi135=iscsi.a;count:1 2024-09-11T08:06:46.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:46 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:46.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:46 smithi122 ceph-mon[39726]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-11T08:06:46.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:46 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/3983672372' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-11T08:06:46.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:46 smithi122 ceph-mon[39726]: osdmap e58: 8 total, 8 up, 8 in 2024-09-11T08:06:46.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:46 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/1005381998' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T08:06:46.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:46 smithi122 ceph-mon[39726]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T08:06:46.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:46 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/3983672372' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T08:06:46.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:46 smithi122 ceph-mon[39726]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.135", "placement": "1;smithi135=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:06:46.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:46 smithi122 ceph-mon[39726]: Saving service iscsi.datapool spec with placement smithi135=iscsi.a;count:1 2024-09-11T08:06:46.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:46 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:47.872 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:06:47.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:47 smithi135 ceph-mon[32689]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-11T08:06:47.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:47 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/3983672372' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-11T08:06:47.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:47 smithi135 ceph-mon[32689]: osdmap e59: 8 total, 8 up, 8 in 2024-09-11T08:06:47.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:47 smithi135 ceph-mon[32689]: pgmap v182: 132 pgs: 3 creating+activating, 16 creating+peering, 23 unknown, 90 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 4.0 KiB/s rd, 1023 B/s wr, 5 op/s 2024-09-11T08:06:47.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:47 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:47.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:47 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:47.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:47 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:47.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:47 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-09-11T08:06:47.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:47 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-11T08:06:47.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:47 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:06:47.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:47 smithi135 ceph-mon[32689]: Deploying daemon iscsi.iscsi.a on smithi135 2024-09-11T08:06:47.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:47 smithi122 ceph-mon[28992]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-11T08:06:47.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:47 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/3983672372' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-11T08:06:47.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:47 smithi122 ceph-mon[28992]: osdmap e59: 8 total, 8 up, 8 in 2024-09-11T08:06:47.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:47 smithi122 ceph-mon[28992]: pgmap v182: 132 pgs: 3 creating+activating, 16 creating+peering, 23 unknown, 90 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 4.0 KiB/s rd, 1023 B/s wr, 5 op/s 2024-09-11T08:06:47.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:47 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:47.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:47 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:47.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:47 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:47.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:47 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-09-11T08:06:47.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:47 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-11T08:06:47.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:47 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:06:47.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:47 smithi122 ceph-mon[28992]: Deploying daemon iscsi.iscsi.a on smithi135 2024-09-11T08:06:47.986 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:47 smithi122 ceph-mon[39726]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-11T08:06:47.986 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:47 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/3983672372' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-11T08:06:47.986 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:47 smithi122 ceph-mon[39726]: osdmap e59: 8 total, 8 up, 8 in 2024-09-11T08:06:47.986 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:47 smithi122 ceph-mon[39726]: pgmap v182: 132 pgs: 3 creating+activating, 16 creating+peering, 23 unknown, 90 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 4.0 KiB/s rd, 1023 B/s wr, 5 op/s 2024-09-11T08:06:47.986 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:47 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:47.986 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:47 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:47.987 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:47 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:47.987 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:47 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-09-11T08:06:47.987 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:47 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-11T08:06:47.987 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:47 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:06:47.987 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:47 smithi122 ceph-mon[39726]: Deploying daemon iscsi.iscsi.a on smithi135 2024-09-11T08:06:48.888 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:48 smithi135 ceph-mon[32689]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-11T08:06:48.888 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:48 smithi135 ceph-mon[32689]: Cluster is now healthy 2024-09-11T08:06:48.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:48 smithi122 ceph-mon[28992]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-11T08:06:48.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:48 smithi122 ceph-mon[28992]: Cluster is now healthy 2024-09-11T08:06:48.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:48 smithi122 ceph-mon[39726]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-11T08:06:48.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:48 smithi122 ceph-mon[39726]: Cluster is now healthy 2024-09-11T08:06:49.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:49 smithi135 ceph-mon[32689]: pgmap v183: 132 pgs: 9 creating+peering, 9 unknown, 114 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 26 KiB/s rd, 2.6 KiB/s wr, 61 op/s 2024-09-11T08:06:49.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:49 smithi122 ceph-mon[28992]: pgmap v183: 132 pgs: 9 creating+peering, 9 unknown, 114 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 26 KiB/s rd, 2.6 KiB/s wr, 61 op/s 2024-09-11T08:06:49.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:49 smithi122 ceph-mon[39726]: pgmap v183: 132 pgs: 9 creating+peering, 9 unknown, 114 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 26 KiB/s rd, 2.6 KiB/s wr, 61 op/s 2024-09-11T08:06:50.011 INFO:journalctl@ceph.iscsi.iscsi.a.smithi135.stdout:Sep 11 08:06:49 smithi135 podman[59620]: 2024-09-11 08:06:49.885487485 +0000 UTC m=+0.100116579 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T08:06:50.143 INFO:teuthology.orchestra.run.smithi135.stdout:Scheduled prometheus update... 2024-09-11T08:06:50.431 INFO:journalctl@ceph.iscsi.iscsi.a.smithi135.stdout:Sep 11 08:06:50 smithi135 podman[59620]: 2024-09-11 08:06:50.152909893 +0000 UTC m=+0.367538976 container create 004c0dd24ec1098cbefae17ab5bab28ab4bc593b0ff44272fad4e38e58eae639 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-11T08:06:50.719 INFO:journalctl@ceph.iscsi.iscsi.a.smithi135.stdout:Sep 11 08:06:50 smithi135 podman[59620]: 2024-09-11 08:06:50.556540358 +0000 UTC m=+0.771169453 container init 004c0dd24ec1098cbefae17ab5bab28ab4bc593b0ff44272fad4e38e58eae639 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, ceph=True) 2024-09-11T08:06:50.720 INFO:journalctl@ceph.iscsi.iscsi.a.smithi135.stdout:Sep 11 08:06:50 smithi135 podman[59620]: 2024-09-11 08:06:50.560073805 +0000 UTC m=+0.774702887 container start 004c0dd24ec1098cbefae17ab5bab28ab4bc593b0ff44272fad4e38e58eae639 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828) 2024-09-11T08:06:50.720 INFO:journalctl@ceph.iscsi.iscsi.a.smithi135.stdout:Sep 11 08:06:50 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu[59688]: log file path now is '/var/log/tcmu-runner.log' 2024-09-11T08:06:50.720 INFO:journalctl@ceph.iscsi.iscsi.a.smithi135.stdout:Sep 11 08:06:50 smithi135 bash[59620]: 004c0dd24ec1098cbefae17ab5bab28ab4bc593b0ff44272fad4e38e58eae639 2024-09-11T08:06:51.092 DEBUG:teuthology.orchestra.run.smithi135:prometheus.a> sudo journalctl -f -n 0 -u ceph-f1d75850-7013-11ef-bcea-c7b262605968@prometheus.a.service 2024-09-11T08:06:51.095 INFO:tasks.cephadm:Adding node-exporter.a on smithi122 2024-09-11T08:06:51.095 INFO:tasks.cephadm:Adding node-exporter.b on smithi135 2024-09-11T08:06:51.095 DEBUG:teuthology.orchestra.run.smithi135:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f1d75850-7013-11ef-bcea-c7b262605968 -- ceph orch apply node-exporter '2;smithi122=a;smithi135=b' 2024-09-11T08:06:51.335 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:51 smithi135 ceph-mon[32689]: from='client.24397 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi135=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:06:51.335 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:51 smithi135 ceph-mon[32689]: Saving service prometheus spec with placement smithi135=a;count:1 2024-09-11T08:06:51.335 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:51 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:51.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:51 smithi122 ceph-mon[28992]: from='client.24397 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi135=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:06:51.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:51 smithi122 ceph-mon[28992]: Saving service prometheus spec with placement smithi135=a;count:1 2024-09-11T08:06:51.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:51 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:51.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:51 smithi122 ceph-mon[39726]: from='client.24397 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi135=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:06:51.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:51 smithi122 ceph-mon[39726]: Saving service prometheus spec with placement smithi135=a;count:1 2024-09-11T08:06:51.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:51 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:51.671 INFO:journalctl@ceph.iscsi.iscsi.a.smithi135.stdout:Sep 11 08:06:51 smithi135 podman[59772]: 2024-09-11 08:06:51.236829996 +0000 UTC m=+0.017342855 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T08:06:51.671 INFO:journalctl@ceph.iscsi.iscsi.a.smithi135.stdout:Sep 11 08:06:51 smithi135 podman[59772]: 2024-09-11 08:06:51.368651135 +0000 UTC m=+0.149163979 container create ff21bbb86657bdc71dd8d1591124429e7b892537148a5029545a1c52dcb6e24b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T08:06:51.671 INFO:journalctl@ceph.iscsi.iscsi.a.smithi135.stdout:Sep 11 08:06:51 smithi135 podman[59772]: 2024-09-11 08:06:51.579086439 +0000 UTC m=+0.359599284 container init ff21bbb86657bdc71dd8d1591124429e7b892537148a5029545a1c52dcb6e24b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-11T08:06:51.671 INFO:journalctl@ceph.iscsi.iscsi.a.smithi135.stdout:Sep 11 08:06:51 smithi135 podman[59772]: 2024-09-11 08:06:51.582467645 +0000 UTC m=+0.362980489 container start ff21bbb86657bdc71dd8d1591124429e7b892537148a5029545a1c52dcb6e24b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-11T08:06:51.757 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:06:51.978 INFO:journalctl@ceph.iscsi.iscsi.a.smithi135.stdout:Sep 11 08:06:51 smithi135 bash[59772]: ff21bbb86657bdc71dd8d1591124429e7b892537148a5029545a1c52dcb6e24b 2024-09-11T08:06:51.978 INFO:journalctl@ceph.iscsi.iscsi.a.smithi135.stdout:Sep 11 08:06:51 smithi135 systemd[1]: Started Ceph iscsi.iscsi.a for f1d75850-7013-11ef-bcea-c7b262605968. 2024-09-11T08:06:51.978 INFO:journalctl@ceph.iscsi.iscsi.a.smithi135.stdout:Sep 11 08:06:51 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[59799]: debug Started the configuration object watcher 2024-09-11T08:06:51.978 INFO:journalctl@ceph.iscsi.iscsi.a.smithi135.stdout:Sep 11 08:06:51 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[59799]: debug Checking for config object changes every 1s 2024-09-11T08:06:51.978 INFO:journalctl@ceph.iscsi.iscsi.a.smithi135.stdout:Sep 11 08:06:51 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[59799]: debug Processing osd blocklist entries for this node 2024-09-11T08:06:52.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:52 smithi135 ceph-mon[32689]: pgmap v184: 132 pgs: 132 active+clean; 581 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 63 KiB/s rd, 4.6 KiB/s wr, 152 op/s 2024-09-11T08:06:52.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:52 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:52.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:52 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:52.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:52 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:52.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:52 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:52.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:52 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:06:52.479 INFO:journalctl@ceph.iscsi.iscsi.a.smithi135.stdout:Sep 11 08:06:52 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[59799]: debug Reading the configuration object to update local LIO configuration 2024-09-11T08:06:52.479 INFO:journalctl@ceph.iscsi.iscsi.a.smithi135.stdout:Sep 11 08:06:52 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[59799]: debug Configuration does not have an entry for this host(smithi135.front.sepia.ceph.com) - nothing to define to LIO 2024-09-11T08:06:52.479 INFO:journalctl@ceph.iscsi.iscsi.a.smithi135.stdout:Sep 11 08:06:52 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[59799]: * Serving Flask app 'rbd-target-api' (lazy loading) 2024-09-11T08:06:52.479 INFO:journalctl@ceph.iscsi.iscsi.a.smithi135.stdout:Sep 11 08:06:52 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[59799]: * Environment: production 2024-09-11T08:06:52.479 INFO:journalctl@ceph.iscsi.iscsi.a.smithi135.stdout:Sep 11 08:06:52 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[59799]: WARNING: This is a development server. Do not use it in a production deployment. 2024-09-11T08:06:52.479 INFO:journalctl@ceph.iscsi.iscsi.a.smithi135.stdout:Sep 11 08:06:52 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[59799]: Use a production WSGI server instead. 2024-09-11T08:06:52.480 INFO:journalctl@ceph.iscsi.iscsi.a.smithi135.stdout:Sep 11 08:06:52 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[59799]: * Debug mode: off 2024-09-11T08:06:52.480 INFO:journalctl@ceph.iscsi.iscsi.a.smithi135.stdout:Sep 11 08:06:52 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[59799]: debug * Running on all addresses. 2024-09-11T08:06:52.480 INFO:journalctl@ceph.iscsi.iscsi.a.smithi135.stdout:Sep 11 08:06:52 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[59799]: WARNING: This is a development server. Do not use it in a production deployment. 2024-09-11T08:06:52.480 INFO:journalctl@ceph.iscsi.iscsi.a.smithi135.stdout:Sep 11 08:06:52 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[59799]: * Running on all addresses. 2024-09-11T08:06:52.480 INFO:journalctl@ceph.iscsi.iscsi.a.smithi135.stdout:Sep 11 08:06:52 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[59799]: WARNING: This is a development server. Do not use it in a production deployment. 2024-09-11T08:06:52.480 INFO:journalctl@ceph.iscsi.iscsi.a.smithi135.stdout:Sep 11 08:06:52 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[59799]: debug * Running on http://[fe80::ec4:7aff:fe88:8083]:5000/ (Press CTRL+C to quit) 2024-09-11T08:06:52.480 INFO:journalctl@ceph.iscsi.iscsi.a.smithi135.stdout:Sep 11 08:06:52 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[59799]: * Running on http://[fe80::ec4:7aff:fe88:8083]:5000/ (Press CTRL+C to quit) 2024-09-11T08:06:52.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:52 smithi122 ceph-mon[39726]: pgmap v184: 132 pgs: 132 active+clean; 581 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 63 KiB/s rd, 4.6 KiB/s wr, 152 op/s 2024-09-11T08:06:52.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:52 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:52.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:52 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:52.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:52 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:52.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:52 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:52.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:52 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:06:52.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:52 smithi122 ceph-mon[28992]: pgmap v184: 132 pgs: 132 active+clean; 581 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 63 KiB/s rd, 4.6 KiB/s wr, 152 op/s 2024-09-11T08:06:52.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:52 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:52.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:52 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:52.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:52 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:52.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:52 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:52.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:52 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:06:53.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:53 smithi135 ceph-mon[32689]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-11T08:06:53.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:53 smithi135 ceph-mon[32689]: from='client.? 172.21.15.135:0/2741646999' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-11T08:06:53.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:53 smithi122 ceph-mon[28992]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-11T08:06:53.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:53 smithi122 ceph-mon[28992]: from='client.? 172.21.15.135:0/2741646999' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-11T08:06:53.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:53 smithi122 ceph-mon[39726]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-11T08:06:53.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:53 smithi122 ceph-mon[39726]: from='client.? 172.21.15.135:0/2741646999' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-11T08:06:53.555 INFO:teuthology.orchestra.run.smithi135.stdout:Scheduled node-exporter update... 2024-09-11T08:06:54.162 DEBUG:teuthology.orchestra.run.smithi122:node-exporter.a> sudo journalctl -f -n 0 -u ceph-f1d75850-7013-11ef-bcea-c7b262605968@node-exporter.a.service 2024-09-11T08:06:54.165 DEBUG:teuthology.orchestra.run.smithi135:node-exporter.b> sudo journalctl -f -n 0 -u ceph-f1d75850-7013-11ef-bcea-c7b262605968@node-exporter.b.service 2024-09-11T08:06:54.168 INFO:tasks.cephadm:Adding alertmanager.a on smithi122 2024-09-11T08:06:54.168 DEBUG:teuthology.orchestra.run.smithi135:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f1d75850-7013-11ef-bcea-c7b262605968 -- ceph orch apply alertmanager '1;smithi122=a' 2024-09-11T08:06:54.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:54 smithi135 ceph-mon[32689]: pgmap v185: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 59 KiB/s rd, 4.5 KiB/s wr, 141 op/s 2024-09-11T08:06:54.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:54 smithi135 ceph-mon[32689]: mgrmap e16: y(active, since 4m), standbys: x 2024-09-11T08:06:54.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:54 smithi135 ceph-mon[32689]: osdmap e60: 8 total, 8 up, 8 in 2024-09-11T08:06:54.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:54 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:54.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:54 smithi122 ceph-mon[39726]: pgmap v185: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 59 KiB/s rd, 4.5 KiB/s wr, 141 op/s 2024-09-11T08:06:54.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:54 smithi122 ceph-mon[39726]: mgrmap e16: y(active, since 4m), standbys: x 2024-09-11T08:06:54.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:54 smithi122 ceph-mon[39726]: osdmap e60: 8 total, 8 up, 8 in 2024-09-11T08:06:54.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:54 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:54.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:54 smithi122 ceph-mon[28992]: pgmap v185: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 59 KiB/s rd, 4.5 KiB/s wr, 141 op/s 2024-09-11T08:06:54.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:54 smithi122 ceph-mon[28992]: mgrmap e16: y(active, since 4m), standbys: x 2024-09-11T08:06:54.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:54 smithi122 ceph-mon[28992]: osdmap e60: 8 total, 8 up, 8 in 2024-09-11T08:06:54.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:54 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:54.587 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:06:55.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:55 smithi135 ceph-mon[32689]: from='client.24433 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi122=a;smithi135=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:06:55.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:55 smithi135 ceph-mon[32689]: Saving service node-exporter spec with placement smithi122=a;smithi135=b;count:2 2024-09-11T08:06:55.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:55 smithi122 ceph-mon[39726]: from='client.24433 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi122=a;smithi135=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:06:55.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:55 smithi122 ceph-mon[39726]: Saving service node-exporter spec with placement smithi122=a;smithi135=b;count:2 2024-09-11T08:06:55.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:55 smithi122 ceph-mon[28992]: from='client.24433 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi122=a;smithi135=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:06:55.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:55 smithi122 ceph-mon[28992]: Saving service node-exporter spec with placement smithi122=a;smithi135=b;count:2 2024-09-11T08:06:56.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:56 smithi135 ceph-mon[32689]: pgmap v187: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 57 KiB/s rd, 4.3 KiB/s wr, 138 op/s 2024-09-11T08:06:56.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:56 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:56.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:56 smithi122 ceph-mon[28992]: pgmap v187: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 57 KiB/s rd, 4.3 KiB/s wr, 138 op/s 2024-09-11T08:06:56.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:56 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:56.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:56 smithi122 ceph-mon[39726]: pgmap v187: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 57 KiB/s rd, 4.3 KiB/s wr, 138 op/s 2024-09-11T08:06:56.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:56 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:56.582 INFO:teuthology.orchestra.run.smithi135.stdout:Scheduled alertmanager update... 2024-09-11T08:06:57.271 DEBUG:teuthology.orchestra.run.smithi122:alertmanager.a> sudo journalctl -f -n 0 -u ceph-f1d75850-7013-11ef-bcea-c7b262605968@alertmanager.a.service 2024-09-11T08:06:57.274 INFO:tasks.cephadm:Adding grafana.a on smithi135 2024-09-11T08:06:57.274 DEBUG:teuthology.orchestra.run.smithi135:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f1d75850-7013-11ef-bcea-c7b262605968 -- ceph orch apply grafana '1;smithi135=a' 2024-09-11T08:06:57.758 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:06:57.879 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:57 smithi135 ceph-mon[32689]: from='client.24439 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi122=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:06:57.880 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:57 smithi135 ceph-mon[32689]: Saving service alertmanager spec with placement smithi122=a;count:1 2024-09-11T08:06:57.880 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:57 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:57.880 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:57 smithi135 ceph-mon[32689]: pgmap v188: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 47 KiB/s rd, 3.6 KiB/s wr, 113 op/s 2024-09-11T08:06:57.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:57 smithi122 ceph-mon[39726]: from='client.24439 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi122=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:06:57.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:57 smithi122 ceph-mon[39726]: Saving service alertmanager spec with placement smithi122=a;count:1 2024-09-11T08:06:57.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:57 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:57.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:57 smithi122 ceph-mon[39726]: pgmap v188: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 47 KiB/s rd, 3.6 KiB/s wr, 113 op/s 2024-09-11T08:06:57.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:57 smithi122 ceph-mon[28992]: from='client.24439 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi122=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:06:57.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:57 smithi122 ceph-mon[28992]: Saving service alertmanager spec with placement smithi122=a;count:1 2024-09-11T08:06:57.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:57 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:57.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:57 smithi122 ceph-mon[28992]: pgmap v188: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 47 KiB/s rd, 3.6 KiB/s wr, 113 op/s 2024-09-11T08:06:59.074 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:58 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:59.074 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:58 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:59.074 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:58 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:06:59.074 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:58 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:06:59.074 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:58 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:59.075 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:58 smithi135 ceph-mon[32689]: Deploying daemon prometheus.a on smithi135 2024-09-11T08:06:59.234 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:58 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:59.234 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:58 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:59.234 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:58 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:06:59.234 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:58 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:06:59.234 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:58 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:59.234 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:58 smithi122 ceph-mon[28992]: Deploying daemon prometheus.a on smithi135 2024-09-11T08:06:59.235 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:58 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:59.235 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:58 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:59.235 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:58 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:06:59.235 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:58 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:06:59.235 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:58 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:06:59.235 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:58 smithi122 ceph-mon[39726]: Deploying daemon prometheus.a on smithi135 2024-09-11T08:06:59.577 INFO:teuthology.orchestra.run.smithi135.stdout:Scheduled grafana update... 2024-09-11T08:07:00.154 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:59 smithi122 ceph-mon[28992]: pgmap v189: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 35 KiB/s rd, 2.6 KiB/s wr, 83 op/s 2024-09-11T08:07:00.154 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:06:59 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:07:00.154 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:59 smithi122 ceph-mon[39726]: pgmap v189: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 35 KiB/s rd, 2.6 KiB/s wr, 83 op/s 2024-09-11T08:07:00.154 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:06:59 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:07:00.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:59 smithi135 ceph-mon[32689]: pgmap v189: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 35 KiB/s rd, 2.6 KiB/s wr, 83 op/s 2024-09-11T08:07:00.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:06:59 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:07:00.455 DEBUG:teuthology.orchestra.run.smithi135:grafana.a> sudo journalctl -f -n 0 -u ceph-f1d75850-7013-11ef-bcea-c7b262605968@grafana.a.service 2024-09-11T08:07:00.457 INFO:tasks.cephadm:Setting up client nodes... 2024-09-11T08:07:00.458 DEBUG:teuthology.orchestra.run.smithi122:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f1d75850-7013-11ef-bcea-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-11T08:07:00.716 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:07:01.083 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:00 smithi122 ceph-mon[28992]: from='client.24445 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi135=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:07:01.083 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:00 smithi122 ceph-mon[28992]: Saving service grafana spec with placement smithi135=a;count:1 2024-09-11T08:07:01.084 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:00 smithi122 ceph-mon[39726]: from='client.24445 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi135=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:07:01.084 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:00 smithi122 ceph-mon[39726]: Saving service grafana spec with placement smithi135=a;count:1 2024-09-11T08:07:01.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:00 smithi135 ceph-mon[32689]: from='client.24445 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi135=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:07:01.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:00 smithi135 ceph-mon[32689]: Saving service grafana spec with placement smithi135=a;count:1 2024-09-11T08:07:02.172 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:01 smithi122 ceph-mon[28992]: pgmap v190: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 9.2 KiB/s rd, 921 B/s wr, 22 op/s 2024-09-11T08:07:02.172 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:01 smithi122 ceph-mon[39726]: pgmap v190: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 9.2 KiB/s rd, 921 B/s wr, 22 op/s 2024-09-11T08:07:02.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:01 smithi135 ceph-mon[32689]: pgmap v190: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 9.2 KiB/s rd, 921 B/s wr, 22 op/s 2024-09-11T08:07:02.228 INFO:journalctl@ceph.iscsi.iscsi.a.smithi135.stdout:Sep 11 08:07:01 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[59799]: debug there is no tcmu-runner data available 2024-09-11T08:07:02.935 INFO:teuthology.orchestra.run.smithi122.stdout:[client.0] 2024-09-11T08:07:02.935 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQCmT+FmMCZqNxAAJDYul2yze6awjYkKXam6EQ== 2024-09-11T08:07:02.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:02 smithi135 ceph-mon[32689]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:07:03.234 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:02 smithi122 ceph-mon[28992]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:07:03.234 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:02 smithi122 ceph-mon[39726]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:07:03.722 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-11T08:07:03.722 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-11T08:07:03.722 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-11T08:07:03.765 DEBUG:teuthology.orchestra.run.smithi135:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f1d75850-7013-11ef-bcea-c7b262605968 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-11T08:07:03.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:03 smithi122 ceph-mon[39726]: pgmap v191: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 102 B/s wr, 1 op/s 2024-09-11T08:07:03.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:03 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/2666681329' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T08:07:03.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:03 smithi122 ceph-mon[39726]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T08:07:03.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:03 smithi122 ceph-mon[39726]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-11T08:07:03.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:03 smithi122 ceph-mon[28992]: pgmap v191: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 102 B/s wr, 1 op/s 2024-09-11T08:07:03.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:03 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/2666681329' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T08:07:03.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:03 smithi122 ceph-mon[28992]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T08:07:03.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:03 smithi122 ceph-mon[28992]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-11T08:07:04.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:03 smithi135 ceph-mon[32689]: pgmap v191: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 102 B/s wr, 1 op/s 2024-09-11T08:07:04.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:03 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/2666681329' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T08:07:04.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:03 smithi135 ceph-mon[32689]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T08:07:04.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:03 smithi135 ceph-mon[32689]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-11T08:07:05.080 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:07:05.998 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:05 smithi135 ceph-mon[32689]: pgmap v192: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 901 B/s rd, 90 B/s wr, 1 op/s 2024-09-11T08:07:06.234 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:05 smithi122 ceph-mon[28992]: pgmap v192: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 901 B/s rd, 90 B/s wr, 1 op/s 2024-09-11T08:07:06.234 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:05 smithi122 ceph-mon[39726]: pgmap v192: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 901 B/s rd, 90 B/s wr, 1 op/s 2024-09-11T08:07:08.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:07 smithi135 ceph-mon[32689]: pgmap v193: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 85 B/s wr, 1 op/s 2024-09-11T08:07:08.234 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:07 smithi122 ceph-mon[28992]: pgmap v193: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 85 B/s wr, 1 op/s 2024-09-11T08:07:08.234 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:07 smithi122 ceph-mon[39726]: pgmap v193: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 85 B/s wr, 1 op/s 2024-09-11T08:07:09.930 INFO:teuthology.orchestra.run.smithi135.stdout:[client.1] 2024-09-11T08:07:09.931 INFO:teuthology.orchestra.run.smithi135.stdout: key = AQCtT+FmMN0VNxAAMAMjC3F2VshFN0j4eDnpQQ== 2024-09-11T08:07:09.931 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:09 smithi135 ceph-mon[32689]: pgmap v194: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:07:10.234 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:09 smithi122 ceph-mon[28992]: pgmap v194: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:07:10.234 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:09 smithi122 ceph-mon[39726]: pgmap v194: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:07:10.899 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:10 smithi135 ceph-mon[32689]: from='client.? 172.21.15.135:0/3745156267' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T08:07:10.900 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:10 smithi135 ceph-mon[32689]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T08:07:10.900 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:10 smithi135 ceph-mon[32689]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-11T08:07:10.924 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-09-11T08:07:10.924 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-09-11T08:07:10.924 DEBUG:teuthology.orchestra.run.smithi135:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-11T08:07:10.966 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-11T08:07:10.966 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-11T08:07:10.967 DEBUG:teuthology.orchestra.run.smithi122:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid f1d75850-7013-11ef-bcea-c7b262605968 -- ceph mgr dump --format=json 2024-09-11T08:07:11.228 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:07:11.228 INFO:journalctl@ceph.prometheus.a.smithi135.stdout:Sep 11 08:07:11 smithi135 podman[61416]: 2024-09-11 08:07:11.124665136 +0000 UTC m=+0.123946563 container create efb99764bec219f38300f5bd756e4ef6fe0dd5926762943751747cf03c04188b (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T08:07:11.234 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:10 smithi122 ceph-mon[28992]: from='client.? 172.21.15.135:0/3745156267' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T08:07:11.234 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:10 smithi122 ceph-mon[28992]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T08:07:11.234 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:10 smithi122 ceph-mon[28992]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-11T08:07:11.234 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:10 smithi122 ceph-mon[39726]: from='client.? 172.21.15.135:0/3745156267' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T08:07:11.234 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:10 smithi122 ceph-mon[39726]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T08:07:11.234 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:10 smithi122 ceph-mon[39726]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-11T08:07:11.729 INFO:journalctl@ceph.prometheus.a.smithi135.stdout:Sep 11 08:07:11 smithi135 podman[61416]: 2024-09-11 08:07:11.385227577 +0000 UTC m=+0.384509006 container init efb99764bec219f38300f5bd756e4ef6fe0dd5926762943751747cf03c04188b (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T08:07:11.729 INFO:journalctl@ceph.prometheus.a.smithi135.stdout:Sep 11 08:07:11 smithi135 podman[61416]: 2024-09-11 08:07:11.388046379 +0000 UTC m=+0.387327818 container start efb99764bec219f38300f5bd756e4ef6fe0dd5926762943751747cf03c04188b (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T08:07:11.729 INFO:journalctl@ceph.prometheus.a.smithi135.stdout:Sep 11 08:07:11 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-prometheus-a[61431]: ts=2024-09-11T08:07:11.412Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2024-09-11T08:07:11.729 INFO:journalctl@ceph.prometheus.a.smithi135.stdout:Sep 11 08:07:11 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-prometheus-a[61431]: ts=2024-09-11T08:07:11.412Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2024-09-11T08:07:11.729 INFO:journalctl@ceph.prometheus.a.smithi135.stdout:Sep 11 08:07:11 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-prometheus-a[61431]: ts=2024-09-11T08:07:11.412Z caller=main.go:570 level=info host_details="(Linux 5.14.0-505.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Sep 5 07:54:07 UTC 2024 x86_64 smithi135 (none))" 2024-09-11T08:07:11.729 INFO:journalctl@ceph.prometheus.a.smithi135.stdout:Sep 11 08:07:11 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-prometheus-a[61431]: ts=2024-09-11T08:07:11.412Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-09-11T08:07:11.729 INFO:journalctl@ceph.prometheus.a.smithi135.stdout:Sep 11 08:07:11 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-prometheus-a[61431]: ts=2024-09-11T08:07:11.412Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-09-11T08:07:11.729 INFO:journalctl@ceph.prometheus.a.smithi135.stdout:Sep 11 08:07:11 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-prometheus-a[61431]: ts=2024-09-11T08:07:11.413Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2024-09-11T08:07:11.730 INFO:journalctl@ceph.prometheus.a.smithi135.stdout:Sep 11 08:07:11 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-prometheus-a[61431]: ts=2024-09-11T08:07:11.413Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2024-09-11T08:07:11.730 INFO:journalctl@ceph.prometheus.a.smithi135.stdout:Sep 11 08:07:11 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-prometheus-a[61431]: ts=2024-09-11T08:07:11.414Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2024-09-11T08:07:11.730 INFO:journalctl@ceph.prometheus.a.smithi135.stdout:Sep 11 08:07:11 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-prometheus-a[61431]: ts=2024-09-11T08:07:11.414Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-09-11T08:07:11.730 INFO:journalctl@ceph.prometheus.a.smithi135.stdout:Sep 11 08:07:11 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-prometheus-a[61431]: ts=2024-09-11T08:07:11.416Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-09-11T08:07:11.730 INFO:journalctl@ceph.prometheus.a.smithi135.stdout:Sep 11 08:07:11 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-prometheus-a[61431]: ts=2024-09-11T08:07:11.416Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=2.383µs 2024-09-11T08:07:11.730 INFO:journalctl@ceph.prometheus.a.smithi135.stdout:Sep 11 08:07:11 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-prometheus-a[61431]: ts=2024-09-11T08:07:11.416Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-09-11T08:07:11.730 INFO:journalctl@ceph.prometheus.a.smithi135.stdout:Sep 11 08:07:11 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-prometheus-a[61431]: ts=2024-09-11T08:07:11.417Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=0 2024-09-11T08:07:11.730 INFO:journalctl@ceph.prometheus.a.smithi135.stdout:Sep 11 08:07:11 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-prometheus-a[61431]: ts=2024-09-11T08:07:11.417Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=26.044µs wal_replay_duration=205.264µs wbl_replay_duration=139ns total_replay_duration=267.121µs 2024-09-11T08:07:11.730 INFO:journalctl@ceph.prometheus.a.smithi135.stdout:Sep 11 08:07:11 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-prometheus-a[61431]: ts=2024-09-11T08:07:11.417Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2024-09-11T08:07:11.730 INFO:journalctl@ceph.prometheus.a.smithi135.stdout:Sep 11 08:07:11 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-prometheus-a[61431]: ts=2024-09-11T08:07:11.417Z caller=main.go:1029 level=info msg="TSDB started" 2024-09-11T08:07:11.730 INFO:journalctl@ceph.prometheus.a.smithi135.stdout:Sep 11 08:07:11 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-prometheus-a[61431]: ts=2024-09-11T08:07:11.417Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-09-11T08:07:11.731 INFO:journalctl@ceph.prometheus.a.smithi135.stdout:Sep 11 08:07:11 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-prometheus-a[61431]: ts=2024-09-11T08:07:11.432Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=14.853131ms db_storage=734ns remote_storage=999ns web_handler=396ns query_engine=589ns scrape=435.177µs scrape_sd=15.318µs notify=638ns notify_sd=1.401µs rules=14.206542ms tracing=6.393µs 2024-09-11T08:07:11.731 INFO:journalctl@ceph.prometheus.a.smithi135.stdout:Sep 11 08:07:11 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-prometheus-a[61431]: ts=2024-09-11T08:07:11.432Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-09-11T08:07:11.731 INFO:journalctl@ceph.prometheus.a.smithi135.stdout:Sep 11 08:07:11 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-prometheus-a[61431]: ts=2024-09-11T08:07:11.432Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-09-11T08:07:11.731 INFO:journalctl@ceph.prometheus.a.smithi135.stdout:Sep 11 08:07:11 smithi135 bash[61416]: efb99764bec219f38300f5bd756e4ef6fe0dd5926762943751747cf03c04188b 2024-09-11T08:07:11.731 INFO:journalctl@ceph.prometheus.a.smithi135.stdout:Sep 11 08:07:11 smithi135 systemd[1]: Started Ceph prometheus.a for f1d75850-7013-11ef-bcea-c7b262605968. 2024-09-11T08:07:11.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:11 smithi122 ceph-mon[39726]: pgmap v195: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:07:11.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:11 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:07:11.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:11 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:07:11.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:11 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:07:11.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:11 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-11T08:07:11.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:11 smithi122 ceph-mon[28992]: pgmap v195: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:07:11.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:11 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:07:11.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:11 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:07:11.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:11 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:07:11.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:11 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-11T08:07:12.228 INFO:journalctl@ceph.iscsi.iscsi.a.smithi135.stdout:Sep 11 08:07:11 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[59799]: debug there is no tcmu-runner data available 2024-09-11T08:07:12.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:11 smithi135 ceph-mon[32689]: pgmap v195: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:07:12.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:11 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:07:12.229 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:11 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:07:12.229 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:11 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' 2024-09-11T08:07:12.229 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:11 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-11T08:07:12.689 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:07:12 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: ignoring --setuser ceph since I am not root 2024-09-11T08:07:12.689 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:07:12 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: ignoring --setgroup ceph since I am not root 2024-09-11T08:07:12.689 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:07:12 smithi122 ceph-mgr[29221]: -- 172.21.15.122:0/2548679860 <== mon.0 v2:172.21.15.122:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56033c100340 con 0x56033c10c800 2024-09-11T08:07:12.689 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:07:12 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:07:12.687+0000 7fb20998cec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T08:07:12.922 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:07:12 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: ignoring --setuser ceph since I am not root 2024-09-11T08:07:12.922 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:07:12 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: ignoring --setgroup ceph since I am not root 2024-09-11T08:07:12.922 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:07:12 smithi135 ceph-mgr[34558]: -- 172.21.15.135:0/2043804194 <== mon.1 v2:172.21.15.135:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5575b3c24340 con 0x5575b3c31000 2024-09-11T08:07:12.922 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:07:12 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:07:12.681+0000 7f3243779ec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T08:07:12.922 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:07:12 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:07:12.734+0000 7f3243779ec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T08:07:12.922 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:07:12 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:07:12.799+0000 7f3243779ec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T08:07:12.922 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:07:12 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:07:12.920+0000 7f3243779ec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T08:07:12.943 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:12 smithi122 ceph-mon[39726]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:07:12.943 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:12 smithi122 ceph-mon[39726]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-11T08:07:12.943 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:12 smithi122 ceph-mon[39726]: mgrmap e17: y(active, since 4m), standbys: x 2024-09-11T08:07:12.943 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:12 smithi122 ceph-mon[28992]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:07:12.944 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:12 smithi122 ceph-mon[28992]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-11T08:07:12.944 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:12 smithi122 ceph-mon[28992]: mgrmap e17: y(active, since 4m), standbys: x 2024-09-11T08:07:12.944 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:07:12 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:07:12.741+0000 7fb20998cec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T08:07:12.944 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:07:12 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:07:12.809+0000 7fb20998cec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T08:07:12.944 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:07:12 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:07:12.941+0000 7fb20998cec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T08:07:13.228 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:07:13 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:07:13.059+0000 7f3243779ec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T08:07:13.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:12 smithi135 ceph-mon[32689]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:07:13.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:12 smithi135 ceph-mon[32689]: from='mgr.14148 172.21.15.122:0/2893682984' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-11T08:07:13.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:12 smithi135 ceph-mon[32689]: mgrmap e17: y(active, since 4m), standbys: x 2024-09-11T08:07:13.233 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:07:13 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:07:13.081+0000 7fb20998cec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T08:07:13.304 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T08:07:13.574 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:07:13 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:07:13.350+0000 7fb20998cec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T08:07:13.575 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:07:13 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:07:13.410+0000 7fb20998cec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T08:07:13.728 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:07:13 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:07:13.315+0000 7f3243779ec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T08:07:13.728 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:07:13 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:07:13.374+0000 7f3243779ec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T08:07:13.729 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:07:13 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:07:13.536+0000 7f3243779ec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T08:07:13.934 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:07:13 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:07:13.573+0000 7fb20998cec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T08:07:14.015 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":17,"flags":0,"active_gid":14148,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":2390419411},{"type":"v1","addr":"172.21.15.122:6801","nonce":2390419411}]},"active_addr":"172.21.15.122:6801/2390419411","active_change":"2024-09-11T08:02:20.655932+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":24112,"name":"x","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.122:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"cephadm","addrvec":[{"type":"v2","addr":"172.21.15.122:0","nonce":2539303076}]},{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.122:0","nonce":4184778050}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.122:0","nonce":1466127201}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.122:0","nonce":1173647492}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.122:0","nonce":1491925756}]}]} 2024-09-11T08:07:14.020 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-11T08:07:14.020 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-11T08:07:14.020 DEBUG:teuthology.orchestra.run.smithi122:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid f1d75850-7013-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T08:07:14.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:13 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/4261077681' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-11T08:07:14.234 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:13 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/4261077681' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-11T08:07:14.234 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:13 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/4261077681' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-11T08:07:14.280 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:07:14.698 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:07:14 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:07:14.470+0000 7fb20998cec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T08:07:14.698 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:07:14 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:07:14.538+0000 7fb20998cec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T08:07:14.728 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:07:14 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:07:14.384+0000 7f3243779ec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T08:07:14.728 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:07:14 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:07:14.449+0000 7f3243779ec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T08:07:15.444 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:07:15 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:07:15.153+0000 7f3243779ec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T08:07:15.444 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:07:15 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:07:15.264+0000 7f3243779ec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T08:07:15.444 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:07:15 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:07:15.384+0000 7f3243779ec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T08:07:15.549 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:07:15 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:07:15.247+0000 7fb20998cec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T08:07:15.549 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:07:15 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:07:15.360+0000 7fb20998cec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T08:07:15.549 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:07:15 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:07:15.485+0000 7fb20998cec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T08:07:15.549 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:07:15 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:07:15.547+0000 7fb20998cec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T08:07:15.697 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:07:15 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:07:15.442+0000 7f3243779ec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T08:07:15.697 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:07:15 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:07:15.568+0000 7f3243779ec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T08:07:15.697 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:07:15 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:07:15.637+0000 7f3243779ec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T08:07:15.698 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:07:15 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:07:15.696+0000 7f3243779ec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T08:07:15.807 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:07:15 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:07:15.677+0000 7fb20998cec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T08:07:15.807 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:07:15 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:07:15.744+0000 7fb20998cec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T08:07:15.807 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:07:15 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:07:15.805+0000 7fb20998cec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T08:07:15.959 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:07:15 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:07:15.829+0000 7f3243779ec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T08:07:15.959 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:07:15 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:07:15.892+0000 7f3243779ec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T08:07:16.077 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:07:15 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:07:15.941+0000 7fb20998cec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T08:07:16.077 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:07:16 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:07:16.008+0000 7fb20998cec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T08:07:16.171 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T08:07:16.172 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":60,"fsid":"f1d75850-7013-11ef-bcea-c7b262605968","created":"2024-09-11T08:01:33.023519+0000","modified":"2024-09-11T08:06:53.352838+0000","last_up_change":"2024-09-11T08:06:27.186494+0000","last_in_change":"2024-09-11T08:06:09.969108+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T08:04:28.972292+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":".rgw.root","create_time":"2024-09-11T08:06:38.349752+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":3,"pool_name":"datapool","create_time":"2024-09-11T08:06:38.581181+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":56,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-09-11T08:06:39.889857+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-09-11T08:06:41.856087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-09-11T08:06:43.918202+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}}}],"osds":[{"osd":0,"uuid":"17a779cc-5769-4022-a66e-e4ef151dd3ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":578396583},{"type":"v1","addr":"172.21.15.122:6803","nonce":578396583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":578396583},{"type":"v1","addr":"172.21.15.122:6805","nonce":578396583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":578396583},{"type":"v1","addr":"172.21.15.122:6809","nonce":578396583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":578396583},{"type":"v1","addr":"172.21.15.122:6807","nonce":578396583}]},"public_addr":"172.21.15.122:6803/578396583","cluster_addr":"172.21.15.122:6805/578396583","heartbeat_back_addr":"172.21.15.122:6809/578396583","heartbeat_front_addr":"172.21.15.122:6807/578396583","state":["exists","up"]},{"osd":1,"uuid":"7b89946c-b41a-4c21-ba99-69cf6c474d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":3517945576},{"type":"v1","addr":"172.21.15.122:6811","nonce":3517945576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":3517945576},{"type":"v1","addr":"172.21.15.122:6813","nonce":3517945576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":3517945576},{"type":"v1","addr":"172.21.15.122:6817","nonce":3517945576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":3517945576},{"type":"v1","addr":"172.21.15.122:6815","nonce":3517945576}]},"public_addr":"172.21.15.122:6811/3517945576","cluster_addr":"172.21.15.122:6813/3517945576","heartbeat_back_addr":"172.21.15.122:6817/3517945576","heartbeat_front_addr":"172.21.15.122:6815/3517945576","state":["exists","up"]},{"osd":2,"uuid":"aa9a050b-2829-4235-b379-60d327384fa1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":3927725903},{"type":"v1","addr":"172.21.15.122:6819","nonce":3927725903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":3927725903},{"type":"v1","addr":"172.21.15.122:6821","nonce":3927725903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":3927725903},{"type":"v1","addr":"172.21.15.122:6825","nonce":3927725903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":3927725903},{"type":"v1","addr":"172.21.15.122:6823","nonce":3927725903}]},"public_addr":"172.21.15.122:6819/3927725903","cluster_addr":"172.21.15.122:6821/3927725903","heartbeat_back_addr":"172.21.15.122:6825/3927725903","heartbeat_front_addr":"172.21.15.122:6823/3927725903","state":["exists","up"]},{"osd":3,"uuid":"de322300-e371-42f0-801f-8239a2988813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":3186512798},{"type":"v1","addr":"172.21.15.122:6827","nonce":3186512798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":3186512798},{"type":"v1","addr":"172.21.15.122:6829","nonce":3186512798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3186512798},{"type":"v1","addr":"172.21.15.122:6833","nonce":3186512798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":3186512798},{"type":"v1","addr":"172.21.15.122:6831","nonce":3186512798}]},"public_addr":"172.21.15.122:6827/3186512798","cluster_addr":"172.21.15.122:6829/3186512798","heartbeat_back_addr":"172.21.15.122:6833/3186512798","heartbeat_front_addr":"172.21.15.122:6831/3186512798","state":["exists","up"]},{"osd":4,"uuid":"f6ddd756-6005-4794-9ccc-d5d421e1fff6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1495829679},{"type":"v1","addr":"172.21.15.135:6801","nonce":1495829679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1495829679},{"type":"v1","addr":"172.21.15.135:6803","nonce":1495829679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1495829679},{"type":"v1","addr":"172.21.15.135:6807","nonce":1495829679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1495829679},{"type":"v1","addr":"172.21.15.135:6805","nonce":1495829679}]},"public_addr":"172.21.15.135:6801/1495829679","cluster_addr":"172.21.15.135:6803/1495829679","heartbeat_back_addr":"172.21.15.135:6807/1495829679","heartbeat_front_addr":"172.21.15.135:6805/1495829679","state":["exists","up"]},{"osd":5,"uuid":"a97f08b8-4600-4a5f-8fbb-e55ed79a6f4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":2575765344},{"type":"v1","addr":"172.21.15.135:6809","nonce":2575765344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":2575765344},{"type":"v1","addr":"172.21.15.135:6811","nonce":2575765344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":2575765344},{"type":"v1","addr":"172.21.15.135:6815","nonce":2575765344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":2575765344},{"type":"v1","addr":"172.21.15.135:6813","nonce":2575765344}]},"public_addr":"172.21.15.135:6809/2575765344","cluster_addr":"172.21.15.135:6811/2575765344","heartbeat_back_addr":"172.21.15.135:6815/2575765344","heartbeat_front_addr":"172.21.15.135:6813/2575765344","state":["exists","up"]},{"osd":6,"uuid":"4caefe69-cb05-4daf-bc91-26097ab2f1f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":1417277044},{"type":"v1","addr":"172.21.15.135:6817","nonce":1417277044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":1417277044},{"type":"v1","addr":"172.21.15.135:6819","nonce":1417277044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":1417277044},{"type":"v1","addr":"172.21.15.135:6823","nonce":1417277044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":1417277044},{"type":"v1","addr":"172.21.15.135:6821","nonce":1417277044}]},"public_addr":"172.21.15.135:6817/1417277044","cluster_addr":"172.21.15.135:6819/1417277044","heartbeat_back_addr":"172.21.15.135:6823/1417277044","heartbeat_front_addr":"172.21.15.135:6821/1417277044","state":["exists","up"]},{"osd":7,"uuid":"be9c6707-0b7d-444c-b6a2-00a69d926260","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":1015573664},{"type":"v1","addr":"172.21.15.135:6825","nonce":1015573664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":1015573664},{"type":"v1","addr":"172.21.15.135:6827","nonce":1015573664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":1015573664},{"type":"v1","addr":"172.21.15.135:6831","nonce":1015573664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":1015573664},{"type":"v1","addr":"172.21.15.135:6829","nonce":1015573664}]},"public_addr":"172.21.15.135:6825/1015573664","cluster_addr":"172.21.15.135:6827/1015573664","heartbeat_back_addr":"172.21.15.135:6831/1015573664","heartbeat_front_addr":"172.21.15.135:6829/1015573664","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:03:39.962238+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:04:03.628854+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:04:26.828010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:04:50.984767+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:05:13.149808+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:05:37.236520+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:06:01.213253+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:06:25.436767+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.122:0/2755262646":"2024-09-12T08:02:20.655827+0000","172.21.15.122:0/1663411035":"2024-09-12T08:02:20.655827+0000","172.21.15.122:6801/1221292339":"2024-09-12T08:02:20.655827+0000","172.21.15.122:6800/1221292339":"2024-09-12T08:02:20.655827+0000","172.21.15.122:0/1815778193":"2024-09-12T08:02:20.655827+0000","172.21.15.122:0/3783853267":"2024-09-12T08:01:58.413985+0000","172.21.15.122:0/199541964":"2024-09-12T08:01:58.413985+0000","172.21.15.122:6800/4195848883":"2024-09-12T08:01:58.413985+0000","172.21.15.122:6801/4195848883":"2024-09-12T08:01:58.413985+0000","172.21.15.122:0/1932082800":"2024-09-12T08:01:58.413985+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":2,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T08:07:16.223 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:07:15 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:07:15.957+0000 7f3243779ec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T08:07:16.223 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:07:16 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:07:16.038+0000 7f3243779ec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T08:07:16.223 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:07:16 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:07:16.094+0000 7f3243779ec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T08:07:16.223 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:07:16 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:07:16.150+0000 7f3243779ec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T08:07:16.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:16 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/1547113116' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T08:07:16.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:16 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/1547113116' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T08:07:16.484 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:07:16 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:07:16.075+0000 7fb20998cec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T08:07:16.484 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:07:16 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:07:16.158+0000 7fb20998cec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T08:07:16.484 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:07:16 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:07:16.219+0000 7fb20998cec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T08:07:16.484 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:07:16 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:07:16.277+0000 7fb20998cec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T08:07:16.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:16 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/1547113116' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T08:07:16.927 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-11T08:07:16.927 DEBUG:teuthology.orchestra.run.smithi122:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid f1d75850-7013-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T08:07:17.217 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:07:17.227 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:07:16 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:07:16.891+0000 7f3243779ec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T08:07:17.227 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:07:16 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: [11/Sep/2024:08:07:16] ENGINE Bus STARTING 2024-09-11T08:07:17.227 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:07:16 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: CherryPy Checker: 2024-09-11T08:07:17.227 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:07:16 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: The Application mounted at '' has an empty config. 2024-09-11T08:07:17.227 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:07:16 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:07:17.227 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:07:17 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: [11/Sep/2024:08:07:16] ENGINE Serving on http://:::9283 2024-09-11T08:07:17.227 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:07:17 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: [11/Sep/2024:08:07:17] ENGINE Bus STARTED 2024-09-11T08:07:17.228 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:07:17 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:07:17.037+0000 7fb20998cec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T08:07:17.228 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:07:17 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: [11/Sep/2024:08:07:17] ENGINE Bus STARTING 2024-09-11T08:07:17.228 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:07:17 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: CherryPy Checker: 2024-09-11T08:07:17.228 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:07:17 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: The Application mounted at '' has an empty config. 2024-09-11T08:07:17.228 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:07:17 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:07:17.228 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:17 smithi122 ceph-mon[28992]: Standby manager daemon x restarted 2024-09-11T08:07:17.228 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:17 smithi122 ceph-mon[28992]: Standby manager daemon x started 2024-09-11T08:07:17.229 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:17 smithi122 ceph-mon[28992]: from='mgr.? 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-11T08:07:17.229 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:17 smithi122 ceph-mon[28992]: from='mgr.? 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T08:07:17.229 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:17 smithi122 ceph-mon[28992]: from='mgr.? 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-11T08:07:17.229 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:17 smithi122 ceph-mon[28992]: from='mgr.? 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T08:07:17.229 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:17 smithi122 ceph-mon[28992]: Active manager daemon y restarted 2024-09-11T08:07:17.229 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:17 smithi122 ceph-mon[28992]: Activating manager daemon y 2024-09-11T08:07:17.229 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:17 smithi122 ceph-mon[28992]: osdmap e61: 8 total, 8 up, 8 in 2024-09-11T08:07:17.229 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:17 smithi122 ceph-mon[28992]: mgrmap e18: y(active, starting, since 0.00960716s), standbys: x 2024-09-11T08:07:17.229 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:17 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:07:17.229 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:17 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:07:17.229 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:17 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:07:17.229 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:17 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T08:07:17.230 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:17 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T08:07:17.230 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:17 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:07:17.230 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:17 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:07:17.230 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:17 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:07:17.230 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:17 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:07:17.230 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:17 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:07:17.230 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:17 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:07:17.230 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:17 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:07:17.230 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:17 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:07:17.230 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:17 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T08:07:17.230 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:17 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T08:07:17.230 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:17 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T08:07:17.230 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:17 smithi122 ceph-mon[28992]: Manager daemon y is now available 2024-09-11T08:07:17.230 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:17 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:07:17.231 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:17 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:07:17.231 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:17 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:07:17.231 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:17 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-11T08:07:17.231 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:17 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-11T08:07:17.231 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:17 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-11T08:07:17.231 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:17 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-11T08:07:17.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:17 smithi135 ceph-mon[32689]: Standby manager daemon x restarted 2024-09-11T08:07:17.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:17 smithi135 ceph-mon[32689]: Standby manager daemon x started 2024-09-11T08:07:17.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:17 smithi135 ceph-mon[32689]: from='mgr.? 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-11T08:07:17.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:17 smithi135 ceph-mon[32689]: from='mgr.? 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T08:07:17.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:17 smithi135 ceph-mon[32689]: from='mgr.? 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-11T08:07:17.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:17 smithi135 ceph-mon[32689]: from='mgr.? 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T08:07:17.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:17 smithi135 ceph-mon[32689]: Active manager daemon y restarted 2024-09-11T08:07:17.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:17 smithi135 ceph-mon[32689]: Activating manager daemon y 2024-09-11T08:07:17.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:17 smithi135 ceph-mon[32689]: osdmap e61: 8 total, 8 up, 8 in 2024-09-11T08:07:17.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:17 smithi135 ceph-mon[32689]: mgrmap e18: y(active, starting, since 0.00960716s), standbys: x 2024-09-11T08:07:17.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:17 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:07:17.480 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:17 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:07:17.480 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:17 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:07:17.480 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:17 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T08:07:17.480 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:17 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T08:07:17.480 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:17 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:07:17.480 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:17 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:07:17.480 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:17 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:07:17.480 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:17 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:07:17.480 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:17 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:07:17.480 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:17 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:07:17.480 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:17 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:07:17.480 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:17 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:07:17.480 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:17 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T08:07:17.481 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:17 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T08:07:17.481 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:17 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T08:07:17.481 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:17 smithi135 ceph-mon[32689]: Manager daemon y is now available 2024-09-11T08:07:17.481 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:17 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:07:17.481 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:17 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:07:17.481 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:17 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:07:17.481 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:17 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-11T08:07:17.481 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:17 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-11T08:07:17.481 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:17 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-11T08:07:17.481 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:17 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-11T08:07:17.484 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:07:17 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: [11/Sep/2024:08:07:17] ENGINE Serving on http://:::9283 2024-09-11T08:07:17.484 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:07:17 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: [11/Sep/2024:08:07:17] ENGINE Bus STARTED 2024-09-11T08:07:17.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:17 smithi122 ceph-mon[39726]: Standby manager daemon x restarted 2024-09-11T08:07:17.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:17 smithi122 ceph-mon[39726]: Standby manager daemon x started 2024-09-11T08:07:17.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:17 smithi122 ceph-mon[39726]: from='mgr.? 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-11T08:07:17.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:17 smithi122 ceph-mon[39726]: from='mgr.? 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T08:07:17.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:17 smithi122 ceph-mon[39726]: from='mgr.? 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-11T08:07:17.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:17 smithi122 ceph-mon[39726]: from='mgr.? 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T08:07:17.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:17 smithi122 ceph-mon[39726]: Active manager daemon y restarted 2024-09-11T08:07:17.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:17 smithi122 ceph-mon[39726]: Activating manager daemon y 2024-09-11T08:07:17.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:17 smithi122 ceph-mon[39726]: osdmap e61: 8 total, 8 up, 8 in 2024-09-11T08:07:17.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:17 smithi122 ceph-mon[39726]: mgrmap e18: y(active, starting, since 0.00960716s), standbys: x 2024-09-11T08:07:17.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:17 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:07:17.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:17 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:07:17.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:17 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:07:17.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:17 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T08:07:17.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:17 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T08:07:17.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:17 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:07:17.486 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:17 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:07:17.486 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:17 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:07:17.486 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:17 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:07:17.486 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:17 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:07:17.486 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:17 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:07:17.486 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:17 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:07:17.486 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:17 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:07:17.486 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:17 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T08:07:17.486 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:17 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T08:07:17.486 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:17 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T08:07:17.486 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:17 smithi122 ceph-mon[39726]: Manager daemon y is now available 2024-09-11T08:07:17.486 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:17 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:07:17.486 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:17 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:07:17.487 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:17 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:07:17.487 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:17 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-11T08:07:17.487 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:17 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-11T08:07:17.487 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:17 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-11T08:07:17.487 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:17 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-11T08:07:18.195 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:07:17 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: ::ffff:172.21.15.135 - - [11/Sep/2024:08:07:17] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:07:18.832 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:07:18 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:07:18.539+0000 7fb1ac70e640 -1 log_channel(cephadm) log [ERR] : [11/Sep/2024:08:07:18] ENGINE Error in HTTPServer.serve 2024-09-11T08:07:18.833 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:07:18 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: Traceback (most recent call last): 2024-09-11T08:07:18.833 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:07:18 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T08:07:18.833 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:07:18 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: self._connections.run(self.expiration_interval) 2024-09-11T08:07:18.833 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:07:18 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T08:07:18.833 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:07:18 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: self._run(expiration_interval) 2024-09-11T08:07:18.833 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:07:18 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T08:07:18.833 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:07:18 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T08:07:18.833 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:07:18 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T08:07:18.833 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:07:18 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T08:07:18.833 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:07:18 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T08:07:18.834 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:07:18 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: s = self.context.wrap_socket( 2024-09-11T08:07:18.834 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:07:18 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T08:07:18.834 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:07:18 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: return self.sslsocket_class._create( 2024-09-11T08:07:18.834 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:07:18 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T08:07:18.834 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:07:18 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: self.do_handshake() 2024-09-11T08:07:18.834 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:07:18 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T08:07:18.834 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:07:18 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: self._sslobj.do_handshake() 2024-09-11T08:07:18.834 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:07:18 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T08:07:18.834 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:07:18 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:07:19.201 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T08:07:19.201 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":61,"fsid":"f1d75850-7013-11ef-bcea-c7b262605968","created":"2024-09-11T08:01:33.023519+0000","modified":"2024-09-11T08:07:17.041075+0000","last_up_change":"2024-09-11T08:06:27.186494+0000","last_in_change":"2024-09-11T08:06:09.969108+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T08:04:28.972292+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":".rgw.root","create_time":"2024-09-11T08:06:38.349752+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":3,"pool_name":"datapool","create_time":"2024-09-11T08:06:38.581181+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":56,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-09-11T08:06:39.889857+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-09-11T08:06:41.856087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-09-11T08:06:43.918202+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}}}],"osds":[{"osd":0,"uuid":"17a779cc-5769-4022-a66e-e4ef151dd3ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":578396583},{"type":"v1","addr":"172.21.15.122:6803","nonce":578396583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":578396583},{"type":"v1","addr":"172.21.15.122:6805","nonce":578396583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":578396583},{"type":"v1","addr":"172.21.15.122:6809","nonce":578396583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":578396583},{"type":"v1","addr":"172.21.15.122:6807","nonce":578396583}]},"public_addr":"172.21.15.122:6803/578396583","cluster_addr":"172.21.15.122:6805/578396583","heartbeat_back_addr":"172.21.15.122:6809/578396583","heartbeat_front_addr":"172.21.15.122:6807/578396583","state":["exists","up"]},{"osd":1,"uuid":"7b89946c-b41a-4c21-ba99-69cf6c474d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":3517945576},{"type":"v1","addr":"172.21.15.122:6811","nonce":3517945576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":3517945576},{"type":"v1","addr":"172.21.15.122:6813","nonce":3517945576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":3517945576},{"type":"v1","addr":"172.21.15.122:6817","nonce":3517945576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":3517945576},{"type":"v1","addr":"172.21.15.122:6815","nonce":3517945576}]},"public_addr":"172.21.15.122:6811/3517945576","cluster_addr":"172.21.15.122:6813/3517945576","heartbeat_back_addr":"172.21.15.122:6817/3517945576","heartbeat_front_addr":"172.21.15.122:6815/3517945576","state":["exists","up"]},{"osd":2,"uuid":"aa9a050b-2829-4235-b379-60d327384fa1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":3927725903},{"type":"v1","addr":"172.21.15.122:6819","nonce":3927725903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":3927725903},{"type":"v1","addr":"172.21.15.122:6821","nonce":3927725903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":3927725903},{"type":"v1","addr":"172.21.15.122:6825","nonce":3927725903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":3927725903},{"type":"v1","addr":"172.21.15.122:6823","nonce":3927725903}]},"public_addr":"172.21.15.122:6819/3927725903","cluster_addr":"172.21.15.122:6821/3927725903","heartbeat_back_addr":"172.21.15.122:6825/3927725903","heartbeat_front_addr":"172.21.15.122:6823/3927725903","state":["exists","up"]},{"osd":3,"uuid":"de322300-e371-42f0-801f-8239a2988813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":3186512798},{"type":"v1","addr":"172.21.15.122:6827","nonce":3186512798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":3186512798},{"type":"v1","addr":"172.21.15.122:6829","nonce":3186512798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3186512798},{"type":"v1","addr":"172.21.15.122:6833","nonce":3186512798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":3186512798},{"type":"v1","addr":"172.21.15.122:6831","nonce":3186512798}]},"public_addr":"172.21.15.122:6827/3186512798","cluster_addr":"172.21.15.122:6829/3186512798","heartbeat_back_addr":"172.21.15.122:6833/3186512798","heartbeat_front_addr":"172.21.15.122:6831/3186512798","state":["exists","up"]},{"osd":4,"uuid":"f6ddd756-6005-4794-9ccc-d5d421e1fff6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1495829679},{"type":"v1","addr":"172.21.15.135:6801","nonce":1495829679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1495829679},{"type":"v1","addr":"172.21.15.135:6803","nonce":1495829679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1495829679},{"type":"v1","addr":"172.21.15.135:6807","nonce":1495829679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1495829679},{"type":"v1","addr":"172.21.15.135:6805","nonce":1495829679}]},"public_addr":"172.21.15.135:6801/1495829679","cluster_addr":"172.21.15.135:6803/1495829679","heartbeat_back_addr":"172.21.15.135:6807/1495829679","heartbeat_front_addr":"172.21.15.135:6805/1495829679","state":["exists","up"]},{"osd":5,"uuid":"a97f08b8-4600-4a5f-8fbb-e55ed79a6f4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":2575765344},{"type":"v1","addr":"172.21.15.135:6809","nonce":2575765344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":2575765344},{"type":"v1","addr":"172.21.15.135:6811","nonce":2575765344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":2575765344},{"type":"v1","addr":"172.21.15.135:6815","nonce":2575765344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":2575765344},{"type":"v1","addr":"172.21.15.135:6813","nonce":2575765344}]},"public_addr":"172.21.15.135:6809/2575765344","cluster_addr":"172.21.15.135:6811/2575765344","heartbeat_back_addr":"172.21.15.135:6815/2575765344","heartbeat_front_addr":"172.21.15.135:6813/2575765344","state":["exists","up"]},{"osd":6,"uuid":"4caefe69-cb05-4daf-bc91-26097ab2f1f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":1417277044},{"type":"v1","addr":"172.21.15.135:6817","nonce":1417277044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":1417277044},{"type":"v1","addr":"172.21.15.135:6819","nonce":1417277044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":1417277044},{"type":"v1","addr":"172.21.15.135:6823","nonce":1417277044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":1417277044},{"type":"v1","addr":"172.21.15.135:6821","nonce":1417277044}]},"public_addr":"172.21.15.135:6817/1417277044","cluster_addr":"172.21.15.135:6819/1417277044","heartbeat_back_addr":"172.21.15.135:6823/1417277044","heartbeat_front_addr":"172.21.15.135:6821/1417277044","state":["exists","up"]},{"osd":7,"uuid":"be9c6707-0b7d-444c-b6a2-00a69d926260","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":1015573664},{"type":"v1","addr":"172.21.15.135:6825","nonce":1015573664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":1015573664},{"type":"v1","addr":"172.21.15.135:6827","nonce":1015573664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":1015573664},{"type":"v1","addr":"172.21.15.135:6831","nonce":1015573664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":1015573664},{"type":"v1","addr":"172.21.15.135:6829","nonce":1015573664}]},"public_addr":"172.21.15.135:6825/1015573664","cluster_addr":"172.21.15.135:6827/1015573664","heartbeat_back_addr":"172.21.15.135:6831/1015573664","heartbeat_front_addr":"172.21.15.135:6829/1015573664","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:03:39.962238+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:04:03.628854+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:04:26.828010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:04:50.984767+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:05:13.149808+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:05:37.236520+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:06:01.213253+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:06:25.436767+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.122:0/1491925756":"2024-09-12T08:07:17.041024+0000","172.21.15.122:6800/2390419411":"2024-09-12T08:07:17.041024+0000","172.21.15.122:0/1932082800":"2024-09-12T08:01:58.413985+0000","172.21.15.122:6801/4195848883":"2024-09-12T08:01:58.413985+0000","172.21.15.122:6800/4195848883":"2024-09-12T08:01:58.413985+0000","172.21.15.122:0/4184778050":"2024-09-12T08:07:17.041024+0000","172.21.15.122:0/2539303076":"2024-09-12T08:07:17.041024+0000","172.21.15.122:0/199541964":"2024-09-12T08:01:58.413985+0000","172.21.15.122:0/3783853267":"2024-09-12T08:01:58.413985+0000","172.21.15.122:0/1815778193":"2024-09-12T08:02:20.655827+0000","172.21.15.122:6800/1221292339":"2024-09-12T08:02:20.655827+0000","172.21.15.122:6801/1221292339":"2024-09-12T08:02:20.655827+0000","172.21.15.122:6801/2390419411":"2024-09-12T08:07:17.041024+0000","172.21.15.122:0/1663411035":"2024-09-12T08:02:20.655827+0000","172.21.15.122:0/2755262646":"2024-09-12T08:02:20.655827+0000","172.21.15.122:0/1466127201":"2024-09-12T08:07:17.041024+0000","172.21.15.122:0/1173647492":"2024-09-12T08:07:17.041024+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T08:07:19.202 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:19 smithi122 ceph-mon[28992]: mgrmap e19: y(active, since 1.01345s), standbys: x 2024-09-11T08:07:19.202 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:19 smithi122 ceph-mon[28992]: [11/Sep/2024:08:07:18] ENGINE Bus STARTING 2024-09-11T08:07:19.202 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:19 smithi122 ceph-mon[28992]: [11/Sep/2024:08:07:18] ENGINE Serving on https://172.21.15.122:7150 2024-09-11T08:07:19.202 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:19 smithi122 ceph-mon[28992]: [11/Sep/2024:08:07:18] ENGINE Bus STARTED 2024-09-11T08:07:19.202 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:19 smithi122 ceph-mon[28992]: [11/Sep/2024:08:07:18] ENGINE Error in HTTPServer.serve 2024-09-11T08:07:19.202 INFO:journalctl@ceph.mon.a.smithi122.stdout: Traceback (most recent call last): 2024-09-11T08:07:19.202 INFO:journalctl@ceph.mon.a.smithi122.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T08:07:19.202 INFO:journalctl@ceph.mon.a.smithi122.stdout: self._connections.run(self.expiration_interval) 2024-09-11T08:07:19.202 INFO:journalctl@ceph.mon.a.smithi122.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T08:07:19.202 INFO:journalctl@ceph.mon.a.smithi122.stdout: self._run(expiration_interval) 2024-09-11T08:07:19.202 INFO:journalctl@ceph.mon.a.smithi122.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T08:07:19.202 INFO:journalctl@ceph.mon.a.smithi122.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T08:07:19.202 INFO:journalctl@ceph.mon.a.smithi122.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T08:07:19.203 INFO:journalctl@ceph.mon.a.smithi122.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T08:07:19.203 INFO:journalctl@ceph.mon.a.smithi122.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T08:07:19.203 INFO:journalctl@ceph.mon.a.smithi122.stdout: s = self.context.wrap_socket( 2024-09-11T08:07:19.203 INFO:journalctl@ceph.mon.a.smithi122.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T08:07:19.203 INFO:journalctl@ceph.mon.a.smithi122.stdout: return self.sslsocket_class._create( 2024-09-11T08:07:19.203 INFO:journalctl@ceph.mon.a.smithi122.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T08:07:19.203 INFO:journalctl@ceph.mon.a.smithi122.stdout: self.do_handshake() 2024-09-11T08:07:19.203 INFO:journalctl@ceph.mon.a.smithi122.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T08:07:19.203 INFO:journalctl@ceph.mon.a.smithi122.stdout: self._sslobj.do_handshake() 2024-09-11T08:07:19.203 INFO:journalctl@ceph.mon.a.smithi122.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T08:07:19.203 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:19 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:07:19.203 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:19 smithi122 ceph-mon[39726]: mgrmap e19: y(active, since 1.01345s), standbys: x 2024-09-11T08:07:19.204 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:19 smithi122 ceph-mon[39726]: [11/Sep/2024:08:07:18] ENGINE Bus STARTING 2024-09-11T08:07:19.204 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:19 smithi122 ceph-mon[39726]: [11/Sep/2024:08:07:18] ENGINE Serving on https://172.21.15.122:7150 2024-09-11T08:07:19.204 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:19 smithi122 ceph-mon[39726]: [11/Sep/2024:08:07:18] ENGINE Bus STARTED 2024-09-11T08:07:19.204 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:19 smithi122 ceph-mon[39726]: [11/Sep/2024:08:07:18] ENGINE Error in HTTPServer.serve 2024-09-11T08:07:19.204 INFO:journalctl@ceph.mon.c.smithi122.stdout: Traceback (most recent call last): 2024-09-11T08:07:19.204 INFO:journalctl@ceph.mon.c.smithi122.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T08:07:19.204 INFO:journalctl@ceph.mon.c.smithi122.stdout: self._connections.run(self.expiration_interval) 2024-09-11T08:07:19.204 INFO:journalctl@ceph.mon.c.smithi122.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T08:07:19.204 INFO:journalctl@ceph.mon.c.smithi122.stdout: self._run(expiration_interval) 2024-09-11T08:07:19.204 INFO:journalctl@ceph.mon.c.smithi122.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T08:07:19.204 INFO:journalctl@ceph.mon.c.smithi122.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T08:07:19.204 INFO:journalctl@ceph.mon.c.smithi122.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T08:07:19.204 INFO:journalctl@ceph.mon.c.smithi122.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T08:07:19.205 INFO:journalctl@ceph.mon.c.smithi122.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T08:07:19.205 INFO:journalctl@ceph.mon.c.smithi122.stdout: s = self.context.wrap_socket( 2024-09-11T08:07:19.205 INFO:journalctl@ceph.mon.c.smithi122.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T08:07:19.205 INFO:journalctl@ceph.mon.c.smithi122.stdout: return self.sslsocket_class._create( 2024-09-11T08:07:19.205 INFO:journalctl@ceph.mon.c.smithi122.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T08:07:19.205 INFO:journalctl@ceph.mon.c.smithi122.stdout: self.do_handshake() 2024-09-11T08:07:19.205 INFO:journalctl@ceph.mon.c.smithi122.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T08:07:19.205 INFO:journalctl@ceph.mon.c.smithi122.stdout: self._sslobj.do_handshake() 2024-09-11T08:07:19.205 INFO:journalctl@ceph.mon.c.smithi122.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T08:07:19.205 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:19 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:07:19.223 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:19 smithi135 ceph-mon[32689]: mgrmap e19: y(active, since 1.01345s), standbys: x 2024-09-11T08:07:19.224 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:19 smithi135 ceph-mon[32689]: [11/Sep/2024:08:07:18] ENGINE Bus STARTING 2024-09-11T08:07:19.224 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:19 smithi135 ceph-mon[32689]: [11/Sep/2024:08:07:18] ENGINE Serving on https://172.21.15.122:7150 2024-09-11T08:07:19.224 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:19 smithi135 ceph-mon[32689]: [11/Sep/2024:08:07:18] ENGINE Bus STARTED 2024-09-11T08:07:19.224 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:19 smithi135 ceph-mon[32689]: [11/Sep/2024:08:07:18] ENGINE Error in HTTPServer.serve 2024-09-11T08:07:19.224 INFO:journalctl@ceph.mon.b.smithi135.stdout: Traceback (most recent call last): 2024-09-11T08:07:19.224 INFO:journalctl@ceph.mon.b.smithi135.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T08:07:19.224 INFO:journalctl@ceph.mon.b.smithi135.stdout: self._connections.run(self.expiration_interval) 2024-09-11T08:07:19.224 INFO:journalctl@ceph.mon.b.smithi135.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T08:07:19.224 INFO:journalctl@ceph.mon.b.smithi135.stdout: self._run(expiration_interval) 2024-09-11T08:07:19.224 INFO:journalctl@ceph.mon.b.smithi135.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T08:07:19.224 INFO:journalctl@ceph.mon.b.smithi135.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T08:07:19.224 INFO:journalctl@ceph.mon.b.smithi135.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T08:07:19.224 INFO:journalctl@ceph.mon.b.smithi135.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T08:07:19.225 INFO:journalctl@ceph.mon.b.smithi135.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T08:07:19.225 INFO:journalctl@ceph.mon.b.smithi135.stdout: s = self.context.wrap_socket( 2024-09-11T08:07:19.225 INFO:journalctl@ceph.mon.b.smithi135.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T08:07:19.225 INFO:journalctl@ceph.mon.b.smithi135.stdout: return self.sslsocket_class._create( 2024-09-11T08:07:19.225 INFO:journalctl@ceph.mon.b.smithi135.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T08:07:19.225 INFO:journalctl@ceph.mon.b.smithi135.stdout: self.do_handshake() 2024-09-11T08:07:19.225 INFO:journalctl@ceph.mon.b.smithi135.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T08:07:19.225 INFO:journalctl@ceph.mon.b.smithi135.stdout: self._sslobj.do_handshake() 2024-09-11T08:07:19.225 INFO:journalctl@ceph.mon.b.smithi135.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T08:07:19.225 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:19 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:07:19.902 DEBUG:teuthology.orchestra.run.smithi122:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid f1d75850-7013-11ef-bcea-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-11T08:07:19.903 DEBUG:teuthology.orchestra.run.smithi122:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid f1d75850-7013-11ef-bcea-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-11T08:07:19.903 DEBUG:teuthology.orchestra.run.smithi122:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid f1d75850-7013-11ef-bcea-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-11T08:07:19.904 DEBUG:teuthology.orchestra.run.smithi122:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid f1d75850-7013-11ef-bcea-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-11T08:07:19.904 DEBUG:teuthology.orchestra.run.smithi122:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid f1d75850-7013-11ef-bcea-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-11T08:07:19.904 DEBUG:teuthology.orchestra.run.smithi122:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid f1d75850-7013-11ef-bcea-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-11T08:07:19.905 DEBUG:teuthology.orchestra.run.smithi122:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid f1d75850-7013-11ef-bcea-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-11T08:07:19.905 DEBUG:teuthology.orchestra.run.smithi122:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid f1d75850-7013-11ef-bcea-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-11T08:07:20.162 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:20 smithi122 ceph-mon[28992]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:07:20.162 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:20 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/3331044223' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T08:07:20.162 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:20 smithi122 ceph-mon[28992]: mgrmap e20: y(active, since 2s), standbys: x 2024-09-11T08:07:20.164 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:20 smithi122 ceph-mon[39726]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:07:20.164 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:20 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/3331044223' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T08:07:20.164 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:20 smithi122 ceph-mon[39726]: mgrmap e20: y(active, since 2s), standbys: x 2024-09-11T08:07:20.477 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:07:20.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:20 smithi135 ceph-mon[32689]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:07:20.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:20 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/3331044223' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T08:07:20.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:20 smithi135 ceph-mon[32689]: mgrmap e20: y(active, since 2s), standbys: x 2024-09-11T08:07:20.762 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:07:20.762 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:07:21.204 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:07:21.204 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:07:21.204 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:07:21.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:21 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:07:21.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:21 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:07:21.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:21 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:07:21.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:21 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:07:21.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:21 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:07:21.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:21 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:07:21.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:21 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:07:21.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:21 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:07:21.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:21 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:07:21.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:21 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:07:21.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:21 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:07:21.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:21 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:07:21.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:21 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:07:21.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:21 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:07:21.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:21 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:07:21.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:21 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:07:21.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:21 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:07:21.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:21 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:07:21.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:21 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:07:21.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:21 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:07:21.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:21 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:07:21.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:21 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:07:21.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:21 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:07:21.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:21 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:07:21.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:21 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:07:21.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:21 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:07:21.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:21 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:07:21.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:21 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:07:21.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:21 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:07:21.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:21 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:07:21.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:21 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:07:21.486 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:21 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:07:21.486 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:21 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:07:21.486 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:21 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:07:21.486 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:21 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:07:21.486 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:21 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:07:21.486 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:21 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:07:21.486 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:21 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:07:21.486 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:21 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:07:22.228 INFO:journalctl@ceph.iscsi.iscsi.a.smithi135.stdout:Sep 11 08:07:21 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[59799]: debug there is no tcmu-runner data available 2024-09-11T08:07:22.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:22 smithi135 ceph-mon[32689]: Adjusting osd_memory_target on smithi135 to 941.8M 2024-09-11T08:07:22.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:22 smithi135 ceph-mon[32689]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:07:22.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:22 smithi135 ceph-mon[32689]: mgrmap e21: y(active, since 4s), standbys: x 2024-09-11T08:07:22.420 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:22 smithi122 ceph-mon[39726]: Adjusting osd_memory_target on smithi135 to 941.8M 2024-09-11T08:07:22.420 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:22 smithi122 ceph-mon[39726]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:07:22.420 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:22 smithi122 ceph-mon[39726]: mgrmap e21: y(active, since 4s), standbys: x 2024-09-11T08:07:22.421 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:22 smithi122 ceph-mon[28992]: Adjusting osd_memory_target on smithi135 to 941.8M 2024-09-11T08:07:22.421 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:22 smithi122 ceph-mon[28992]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:07:22.421 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:22 smithi122 ceph-mon[28992]: mgrmap e21: y(active, since 4s), standbys: x 2024-09-11T08:07:23.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:23 smithi135 ceph-mon[32689]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:07:23.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:23 smithi122 ceph-mon[39726]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:07:23.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:23 smithi122 ceph-mon[28992]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:07:24.234 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:24 smithi122 ceph-mon[28992]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 35 KiB/s rd, 0 B/s wr, 14 op/s 2024-09-11T08:07:24.234 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:24 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:07:24.234 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:24 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:07:24.234 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:24 smithi122 ceph-mon[39726]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 35 KiB/s rd, 0 B/s wr, 14 op/s 2024-09-11T08:07:24.234 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:24 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:07:24.235 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:24 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:07:24.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:24 smithi135 ceph-mon[32689]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 35 KiB/s rd, 0 B/s wr, 14 op/s 2024-09-11T08:07:24.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:24 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:07:24.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:24 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:07:25.890 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:07:25.890 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:07:26.095 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:25 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:07:26.095 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:25 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:07:26.095 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:25 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:07:26.095 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:25 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:07:26.095 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:25 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:07:26.095 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:25 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:07:26.095 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:25 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:07:26.095 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:25 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:07:26.095 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:25 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:07:26.096 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:25 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:07:26.096 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:25 smithi122 ceph-mon[39726]: Adjusting osd_memory_target on smithi122 to 3759M 2024-09-11T08:07:26.096 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:25 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:07:26.096 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:25 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:07:26.096 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:25 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:07:26.096 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:25 smithi122 ceph-mon[39726]: Updating smithi122:/etc/ceph/ceph.conf 2024-09-11T08:07:26.096 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:25 smithi122 ceph-mon[39726]: Updating smithi135:/etc/ceph/ceph.conf 2024-09-11T08:07:26.096 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:25 smithi122 ceph-mon[39726]: Updating smithi122:/var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:07:26.096 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:25 smithi122 ceph-mon[39726]: Updating smithi135:/var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:07:26.096 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:25 smithi122 ceph-mon[39726]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-11T08:07:26.096 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:25 smithi122 ceph-mon[39726]: Updating smithi122:/etc/ceph/ceph.client.admin.keyring 2024-09-11T08:07:26.096 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:25 smithi122 ceph-mon[39726]: Updating smithi135:/etc/ceph/ceph.client.admin.keyring 2024-09-11T08:07:26.096 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:25 smithi122 ceph-mon[39726]: Updating smithi122:/var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T08:07:26.097 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:25 smithi122 ceph-mon[39726]: Updating smithi135:/var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T08:07:26.097 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:25 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:07:26.097 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:25 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:07:26.097 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:25 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:07:26.097 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:25 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:07:26.097 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:25 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:07:26.097 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:25 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:07:26.097 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:25 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:07:26.097 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:25 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:07:26.097 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:25 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:07:26.097 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:25 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:07:26.098 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:25 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:07:26.098 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:25 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:07:26.098 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:25 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:07:26.098 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:25 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:07:26.098 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:25 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:07:26.098 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:25 smithi122 ceph-mon[28992]: Adjusting osd_memory_target on smithi122 to 3759M 2024-09-11T08:07:26.098 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:25 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:07:26.098 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:25 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:07:26.098 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:25 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:07:26.098 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:25 smithi122 ceph-mon[28992]: Updating smithi122:/etc/ceph/ceph.conf 2024-09-11T08:07:26.098 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:25 smithi122 ceph-mon[28992]: Updating smithi135:/etc/ceph/ceph.conf 2024-09-11T08:07:26.098 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:25 smithi122 ceph-mon[28992]: Updating smithi122:/var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:07:26.098 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:25 smithi122 ceph-mon[28992]: Updating smithi135:/var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:07:26.099 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:25 smithi122 ceph-mon[28992]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-11T08:07:26.099 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:25 smithi122 ceph-mon[28992]: Updating smithi122:/etc/ceph/ceph.client.admin.keyring 2024-09-11T08:07:26.099 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:25 smithi122 ceph-mon[28992]: Updating smithi135:/etc/ceph/ceph.client.admin.keyring 2024-09-11T08:07:26.099 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:25 smithi122 ceph-mon[28992]: Updating smithi122:/var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T08:07:26.099 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:25 smithi122 ceph-mon[28992]: Updating smithi135:/var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T08:07:26.099 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:25 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:07:26.099 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:25 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:07:26.099 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:25 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:07:26.099 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:25 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:07:26.099 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:25 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:07:26.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:25 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:07:26.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:25 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:07:26.229 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:25 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:07:26.229 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:25 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:07:26.229 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:25 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:07:26.229 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:25 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:07:26.229 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:25 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:07:26.229 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:25 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:07:26.229 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:25 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:07:26.229 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:25 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:07:26.229 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:25 smithi135 ceph-mon[32689]: Adjusting osd_memory_target on smithi122 to 3759M 2024-09-11T08:07:26.229 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:25 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:07:26.229 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:25 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:07:26.229 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:25 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:07:26.229 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:25 smithi135 ceph-mon[32689]: Updating smithi122:/etc/ceph/ceph.conf 2024-09-11T08:07:26.230 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:25 smithi135 ceph-mon[32689]: Updating smithi135:/etc/ceph/ceph.conf 2024-09-11T08:07:26.230 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:25 smithi135 ceph-mon[32689]: Updating smithi122:/var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:07:26.230 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:25 smithi135 ceph-mon[32689]: Updating smithi135:/var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:07:26.230 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:25 smithi135 ceph-mon[32689]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-11T08:07:26.230 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:25 smithi135 ceph-mon[32689]: Updating smithi122:/etc/ceph/ceph.client.admin.keyring 2024-09-11T08:07:26.230 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:25 smithi135 ceph-mon[32689]: Updating smithi135:/etc/ceph/ceph.client.admin.keyring 2024-09-11T08:07:26.230 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:25 smithi135 ceph-mon[32689]: Updating smithi122:/var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T08:07:26.230 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:25 smithi135 ceph-mon[32689]: Updating smithi135:/var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T08:07:26.230 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:25 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:07:26.230 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:25 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:07:26.230 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:25 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:07:26.230 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:25 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:07:26.230 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:25 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:07:27.051 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:26 smithi122 ceph-mon[39726]: Deploying daemon alertmanager.a on smithi122 2024-09-11T08:07:27.052 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:26 smithi122 ceph-mon[28992]: Deploying daemon alertmanager.a on smithi122 2024-09-11T08:07:27.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:26 smithi135 ceph-mon[32689]: Deploying daemon alertmanager.a on smithi122 2024-09-11T08:07:28.020 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:27 smithi122 ceph-mon[28992]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-11T08:07:28.021 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:27 smithi122 ceph-mon[39726]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-11T08:07:28.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:27 smithi135 ceph-mon[32689]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-11T08:07:28.228 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:07:27 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: ::ffff:172.21.15.135 - - [11/Sep/2024:08:07:27] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:07:30.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:30 smithi135 ceph-mon[32689]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 17 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-11T08:07:30.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:30 smithi122 ceph-mon[28992]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 17 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-11T08:07:30.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:30 smithi122 ceph-mon[39726]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 17 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-11T08:07:31.978 INFO:journalctl@ceph.iscsi.iscsi.a.smithi135.stdout:Sep 11 08:07:31 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[59799]: debug there is no tcmu-runner data available 2024-09-11T08:07:32.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:32 smithi135 ceph-mon[32689]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-11T08:07:32.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:32 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:07:32.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:32 smithi122 ceph-mon[28992]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-11T08:07:32.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:32 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:07:32.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:32 smithi122 ceph-mon[39726]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-11T08:07:32.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:32 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:07:33.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:33 smithi135 ceph-mon[32689]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:07:33.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:33 smithi122 ceph-mon[28992]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:07:33.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:33 smithi122 ceph-mon[39726]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:07:34.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:34 smithi135 ceph-mon[32689]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 16 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-11T08:07:34.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:34 smithi122 ceph-mon[28992]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 16 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-11T08:07:34.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:34 smithi122 ceph-mon[39726]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 16 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-11T08:07:35.227 INFO:teuthology.orchestra.run.smithi122.stdout:154618822681 2024-09-11T08:07:35.228 DEBUG:teuthology.orchestra.run.smithi122:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid f1d75850-7013-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-11T08:07:36.234 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:36 smithi122 ceph-mon[28992]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:07:36.234 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:36 smithi122 ceph-mon[39726]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:07:36.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:36 smithi135 ceph-mon[32689]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:07:36.816 INFO:teuthology.orchestra.run.smithi122.stdout:206158430223 2024-09-11T08:07:36.817 DEBUG:teuthology.orchestra.run.smithi122:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid f1d75850-7013-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-11T08:07:37.224 INFO:teuthology.orchestra.run.smithi122.stdout:55834574892 2024-09-11T08:07:37.224 DEBUG:teuthology.orchestra.run.smithi122:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid f1d75850-7013-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-11T08:07:38.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:38 smithi135 ceph-mon[32689]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:07:38.228 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:07:37 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: ::ffff:172.21.15.135 - - [11/Sep/2024:08:07:37] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:07:38.234 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:38 smithi122 ceph-mon[28992]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:07:38.234 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:38 smithi122 ceph-mon[39726]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:07:38.586 INFO:teuthology.orchestra.run.smithi122.stdout:180388626452 2024-09-11T08:07:38.586 DEBUG:teuthology.orchestra.run.smithi122:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid f1d75850-7013-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-11T08:07:39.418 INFO:teuthology.orchestra.run.smithi122.stdout:107374182434 2024-09-11T08:07:39.419 DEBUG:teuthology.orchestra.run.smithi122:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid f1d75850-7013-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-11T08:07:40.234 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:40 smithi122 ceph-mon[28992]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:07:40.234 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:40 smithi122 ceph-mon[39726]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:07:40.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:40 smithi135 ceph-mon[32689]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:07:41.636 INFO:teuthology.orchestra.run.smithi122.stdout:128849018910 2024-09-11T08:07:41.636 DEBUG:teuthology.orchestra.run.smithi122:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid f1d75850-7013-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-11T08:07:42.134 INFO:journalctl@ceph.iscsi.iscsi.a.smithi135.stdout:Sep 11 08:07:41 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[59799]: debug there is no tcmu-runner data available 2024-09-11T08:07:42.429 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:42 smithi122 ceph-mon[39726]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:07:42.429 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:42 smithi122 ceph-mon[28992]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:07:42.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:42 smithi135 ceph-mon[32689]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:07:43.411 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:43 smithi122 ceph-mon[28992]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:07:43.412 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:43 smithi122 ceph-mon[39726]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:07:43.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:43 smithi135 ceph-mon[32689]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:07:43.575 INFO:teuthology.orchestra.run.smithi122.stdout:34359738417 2024-09-11T08:07:43.575 DEBUG:teuthology.orchestra.run.smithi122:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid f1d75850-7013-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-11T08:07:43.779 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:07:43.779 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:07:44.234 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:44 smithi122 ceph-mon[28992]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:07:44.234 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:44 smithi122 ceph-mon[39726]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:07:44.349 INFO:teuthology.orchestra.run.smithi122.stdout:77309411368 2024-09-11T08:07:44.349 DEBUG:teuthology.orchestra.run.smithi122:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid f1d75850-7013-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-11T08:07:44.350 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:07:44.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:44 smithi135 ceph-mon[32689]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:07:44.586 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:07:46.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:46 smithi135 ceph-mon[32689]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:07:46.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:46 smithi122 ceph-mon[28992]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:07:46.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:46 smithi122 ceph-mon[39726]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:07:47.414 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:07:47.415 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:07:47.416 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:07:47.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:47 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:07:47.483 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:47 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:07:47.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:47 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:07:48.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:48 smithi135 ceph-mon[32689]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:07:48.228 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:07:47 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: ::ffff:172.21.15.135 - - [11/Sep/2024:08:07:47] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:07:48.384 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:48 smithi122 ceph-mon[28992]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:07:48.385 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:48 smithi122 ceph-mon[39726]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:07:48.412 INFO:teuthology.orchestra.run.smithi122.stdout:55834574895 2024-09-11T08:07:49.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:49 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/4058537661' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-11T08:07:49.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:49 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/4058537661' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-11T08:07:49.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:49 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/4058537661' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-11T08:07:50.358 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:50 smithi122 ceph-mon[28992]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:07:50.358 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:50 smithi122 ceph-mon[39726]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:07:50.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:50 smithi135 ceph-mon[32689]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:07:50.836 INFO:teuthology.orchestra.run.smithi122.stdout:34359738421 2024-09-11T08:07:51.396 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:51 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/2807984971' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-11T08:07:51.396 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:51 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/2807984971' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-11T08:07:51.424 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738417 got 34359738421 for osd.0 2024-09-11T08:07:51.424 DEBUG:teuthology.parallel:result is None 2024-09-11T08:07:51.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:51 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/2807984971' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-11T08:07:51.861 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574892 got 55834574895 for osd.1 2024-09-11T08:07:51.861 DEBUG:teuthology.parallel:result is None 2024-09-11T08:07:52.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:52 smithi135 ceph-mon[32689]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:07:52.229 INFO:journalctl@ceph.iscsi.iscsi.a.smithi135.stdout:Sep 11 08:07:51 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[59799]: debug there is no tcmu-runner data available 2024-09-11T08:07:52.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:52 smithi122 ceph-mon[28992]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:07:52.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:52 smithi122 ceph-mon[39726]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:07:53.234 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:53 smithi122 ceph-mon[28992]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:07:53.234 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:53 smithi122 ceph-mon[39726]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:07:53.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:53 smithi135 ceph-mon[32689]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:07:54.111 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:07:54.414 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:54 smithi122 ceph-mon[28992]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:07:54.414 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:54 smithi122 ceph-mon[39726]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:07:54.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:54 smithi135 ceph-mon[32689]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:07:55.339 INFO:teuthology.orchestra.run.smithi122.stdout:206158430228 2024-09-11T08:07:56.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:56 smithi135 ceph-mon[32689]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:07:56.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:56 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/3187218006' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-11T08:07:56.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:56 smithi122 ceph-mon[28992]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:07:56.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:56 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/3187218006' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-11T08:07:56.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:56 smithi122 ceph-mon[39726]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:07:56.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:56 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/3187218006' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-11T08:07:56.935 INFO:teuthology.orchestra.run.smithi122.stdout:180388626456 2024-09-11T08:07:57.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:57 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/2513381330' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-11T08:07:57.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:57 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/2513381330' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-11T08:07:57.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:57 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/2513381330' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-11T08:07:57.603 INFO:tasks.cephadm.ceph_manager.ceph:need seq 180388626452 got 180388626456 for osd.6 2024-09-11T08:07:57.604 DEBUG:teuthology.parallel:result is None 2024-09-11T08:07:57.936 INFO:teuthology.orchestra.run.smithi122.stdout:128849018914 2024-09-11T08:07:58.190 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:58 smithi122 ceph-mon[39726]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:07:58.190 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:58 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/530389762' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-11T08:07:58.190 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:58 smithi122 ceph-mon[28992]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:07:58.190 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:58 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/530389762' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-11T08:07:58.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:58 smithi135 ceph-mon[32689]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:07:58.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:58 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/530389762' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-11T08:07:58.228 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:07:57 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: ::ffff:172.21.15.135 - - [11/Sep/2024:08:07:57] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:07:58.449 INFO:journalctl@ceph.alertmanager.a.smithi122.stdout:Sep 11 08:07:58 smithi122 systemd[1]: Started Ceph alertmanager.a for f1d75850-7013-11ef-bcea-c7b262605968. 2024-09-11T08:07:58.710 INFO:teuthology.orchestra.run.smithi122.stdout:154618822686 2024-09-11T08:07:58.711 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:07:58 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: [11/Sep/2024:08:07:58] ENGINE Bus STOPPING 2024-09-11T08:07:58.984 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:07:58 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: [11/Sep/2024:08:07:58] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-11T08:07:58.984 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:07:58 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: [11/Sep/2024:08:07:58] ENGINE Bus STOPPED 2024-09-11T08:07:58.984 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:07:58 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: [11/Sep/2024:08:07:58] ENGINE Bus STARTING 2024-09-11T08:07:58.984 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:07:58 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: [11/Sep/2024:08:07:58] ENGINE Serving on http://:::9283 2024-09-11T08:07:58.984 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:07:58 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: [11/Sep/2024:08:07:58] ENGINE Bus STARTED 2024-09-11T08:07:59.245 INFO:tasks.cephadm.ceph_manager.ceph:need seq 128849018910 got 128849018914 for osd.4 2024-09-11T08:07:59.245 DEBUG:teuthology.parallel:result is None 2024-09-11T08:07:59.472 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:59 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:07:59.472 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:59 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:07:59.473 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:59 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:07:59.473 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:59 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:07:59.473 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:59 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:07:59.473 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:59 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:07:59.473 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:59 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-11T08:07:59.473 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:59 smithi122 ceph-mon[28992]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-11T08:07:59.473 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:59 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:07:59.473 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:59 smithi122 ceph-mon[28992]: Deploying daemon grafana.a on smithi135 2024-09-11T08:07:59.473 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:07:59 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/1730917170' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-11T08:07:59.473 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:59 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:07:59.474 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:59 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:07:59.474 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:59 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:07:59.474 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:59 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:07:59.474 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:59 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:07:59.474 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:59 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:07:59.474 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:59 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-11T08:07:59.474 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:59 smithi122 ceph-mon[39726]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-11T08:07:59.474 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:59 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:07:59.474 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:59 smithi122 ceph-mon[39726]: Deploying daemon grafana.a on smithi135 2024-09-11T08:07:59.474 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:07:59 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/1730917170' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-11T08:07:59.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:59 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:07:59.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:59 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:07:59.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:59 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:07:59.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:59 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:07:59.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:59 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:07:59.729 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:59 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:07:59.729 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:59 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-11T08:07:59.729 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:59 smithi135 ceph-mon[32689]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-11T08:07:59.729 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:59 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:07:59.729 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:59 smithi135 ceph-mon[32689]: Deploying daemon grafana.a on smithi135 2024-09-11T08:07:59.729 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:07:59 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/1730917170' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-11T08:07:59.968 INFO:tasks.cephadm.ceph_manager.ceph:need seq 206158430223 got 206158430228 for osd.7 2024-09-11T08:07:59.968 DEBUG:teuthology.parallel:result is None 2024-09-11T08:08:00.304 INFO:teuthology.orchestra.run.smithi122.stdout:77309411372 2024-09-11T08:08:00.305 INFO:journalctl@ceph.alertmanager.a.smithi122.stdout:Sep 11 08:08:00 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-alertmanager-a[71755]: ts=2024-09-11T08:08:00.207Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000420157s 2024-09-11T08:08:00.674 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:00 smithi122 ceph-mon[39726]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:08:00.674 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:00 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/675707334' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-11T08:08:00.675 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:00 smithi122 ceph-mon[28992]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:08:00.675 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:00 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/675707334' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-11T08:08:00.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:00 smithi135 ceph-mon[32689]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:08:00.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:00 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/675707334' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-11T08:08:01.074 INFO:tasks.cephadm.ceph_manager.ceph:need seq 154618822681 got 154618822686 for osd.5 2024-09-11T08:08:01.074 DEBUG:teuthology.parallel:result is None 2024-09-11T08:08:01.562 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411368 got 77309411372 for osd.2 2024-09-11T08:08:01.562 DEBUG:teuthology.parallel:result is None 2024-09-11T08:08:01.959 INFO:teuthology.orchestra.run.smithi122.stdout:107374182440 2024-09-11T08:08:02.228 INFO:journalctl@ceph.iscsi.iscsi.a.smithi135.stdout:Sep 11 08:08:01 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[59799]: debug there is no tcmu-runner data available 2024-09-11T08:08:02.584 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:02 smithi122 ceph-mon[28992]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:08:02.584 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:02 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/1650212584' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-11T08:08:02.584 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:02 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:02.584 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:02 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:08:02.584 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:02 smithi122 ceph-mon[39726]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:08:02.584 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:02 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/1650212584' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-11T08:08:02.585 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:02 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:02.585 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:02 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:08:02.609 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182434 got 107374182440 for osd.3 2024-09-11T08:08:02.609 DEBUG:teuthology.parallel:result is None 2024-09-11T08:08:02.610 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-11T08:08:02.610 DEBUG:teuthology.orchestra.run.smithi122:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid f1d75850-7013-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T08:08:02.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:02 smithi135 ceph-mon[32689]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:08:02.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:02 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/1650212584' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-11T08:08:02.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:02 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:02.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:02 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:08:02.858 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:08:03.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:03 smithi122 ceph-mon[28992]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:08:03.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:03 smithi122 ceph-mon[39726]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:08:03.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:03 smithi135 ceph-mon[32689]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:08:04.555 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:04 smithi122 ceph-mon[39726]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:08:04.555 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:04 smithi122 ceph-mon[28992]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:08:04.707 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T08:08:04.710 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T08:08:04.727 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:04 smithi135 ceph-mon[32689]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:08:05.276 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2024-09-11T08:08:03.068653+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596188,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":930,"num_read_kb":787,"num_write":543,"num_write_kb":1671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":548,"ondisk_log_size":548,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2385872,"kb_used_data":7036,"kb_used_omap":13,"kb_used_meta":2378738,"kb_avail":747575344,"statfs":{"total":767960285184,"available":765517152256,"internally_reserved":0,"allocated":7204864,"data_stored":3992702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2435828416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002837"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.504810+0000","last_change":"2024-09-11T08:06:46.174781+0000","last_active":"2024-09-11T08:06:53.504810+0000","last_peered":"2024-09-11T08:06:53.504810+0000","last_clean":"2024-09-11T08:06:53.504810+0000","last_became_active":"2024-09-11T08:06:46.174393+0000","last_became_peered":"2024-09-11T08:06:46.174393+0000","last_unstale":"2024-09-11T08:06:53.504810+0000","last_undegraded":"2024-09-11T08:06:53.504810+0000","last_fullsized":"2024-09-11T08:06:53.504810+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_clean_scrub_stamp":"2024-09-11T08:06:44.606510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:31:08.409161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.357679+0000","last_change":"2024-09-11T08:06:39.610840+0000","last_active":"2024-09-11T08:06:53.357679+0000","last_peered":"2024-09-11T08:06:53.357679+0000","last_clean":"2024-09-11T08:06:53.357679+0000","last_became_active":"2024-09-11T08:06:39.610598+0000","last_became_peered":"2024-09-11T08:06:39.610598+0000","last_unstale":"2024-09-11T08:06:53.357679+0000","last_undegraded":"2024-09-11T08:06:53.357679+0000","last_fullsized":"2024-09-11T08:06:53.357679+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_clean_scrub_stamp":"2024-09-11T08:06:38.581279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:24:22.740831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"59'15","reported_seq":41,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.505130+0000","last_change":"2024-09-11T08:06:41.612891+0000","last_active":"2024-09-11T08:06:53.505130+0000","last_peered":"2024-09-11T08:06:53.505130+0000","last_clean":"2024-09-11T08:06:53.505130+0000","last_became_active":"2024-09-11T08:06:41.612600+0000","last_became_peered":"2024-09-11T08:06:41.612600+0000","last_unstale":"2024-09-11T08:06:53.505130+0000","last_undegraded":"2024-09-11T08:06:53.505130+0000","last_fullsized":"2024-09-11T08:06:53.505130+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_clean_scrub_stamp":"2024-09-11T08:06:40.591817+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:03:34.211085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.355780+0000","last_change":"2024-09-11T08:06:43.625089+0000","last_active":"2024-09-11T08:06:53.355780+0000","last_peered":"2024-09-11T08:06:53.355780+0000","last_clean":"2024-09-11T08:06:53.355780+0000","last_became_active":"2024-09-11T08:06:43.624550+0000","last_became_peered":"2024-09-11T08:06:43.624550+0000","last_unstale":"2024-09-11T08:06:53.355780+0000","last_undegraded":"2024-09-11T08:06:53.355780+0000","last_fullsized":"2024-09-11T08:06:53.355780+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_clean_scrub_stamp":"2024-09-11T08:06:42.602559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:45:31.954738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.505573+0000","last_change":"2024-09-11T08:06:39.606782+0000","last_active":"2024-09-11T08:06:53.505573+0000","last_peered":"2024-09-11T08:06:53.505573+0000","last_clean":"2024-09-11T08:06:53.505573+0000","last_became_active":"2024-09-11T08:06:39.606496+0000","last_became_peered":"2024-09-11T08:06:39.606496+0000","last_unstale":"2024-09-11T08:06:53.505573+0000","last_undegraded":"2024-09-11T08:06:53.505573+0000","last_fullsized":"2024-09-11T08:06:53.505573+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_clean_scrub_stamp":"2024-09-11T08:06:38.581279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:14:50.505270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"59'9","reported_seq":32,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.355788+0000","last_change":"2024-09-11T08:06:41.615154+0000","last_active":"2024-09-11T08:06:53.355788+0000","last_peered":"2024-09-11T08:06:53.355788+0000","last_clean":"2024-09-11T08:06:53.355788+0000","last_became_active":"2024-09-11T08:06:41.614826+0000","last_became_peered":"2024-09-11T08:06:41.614826+0000","last_unstale":"2024-09-11T08:06:53.355788+0000","last_undegraded":"2024-09-11T08:06:53.355788+0000","last_fullsized":"2024-09-11T08:06:53.355788+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_clean_scrub_stamp":"2024-09-11T08:06:40.591817+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:15:35.543628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.922718+0000","last_change":"2024-09-11T08:06:43.628775+0000","last_active":"2024-09-11T08:06:53.922718+0000","last_peered":"2024-09-11T08:06:53.922718+0000","last_clean":"2024-09-11T08:06:53.922718+0000","last_became_active":"2024-09-11T08:06:43.628471+0000","last_became_peered":"2024-09-11T08:06:43.628471+0000","last_unstale":"2024-09-11T08:06:53.922718+0000","last_undegraded":"2024-09-11T08:06:53.922718+0000","last_fullsized":"2024-09-11T08:06:53.922718+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_clean_scrub_stamp":"2024-09-11T08:06:42.602559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:50:17.482273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.355972+0000","last_change":"2024-09-11T08:06:45.626960+0000","last_active":"2024-09-11T08:06:53.355972+0000","last_peered":"2024-09-11T08:06:53.355972+0000","last_clean":"2024-09-11T08:06:53.355972+0000","last_became_active":"2024-09-11T08:06:45.626701+0000","last_became_peered":"2024-09-11T08:06:45.626701+0000","last_unstale":"2024-09-11T08:06:53.355972+0000","last_undegraded":"2024-09-11T08:06:53.355972+0000","last_fullsized":"2024-09-11T08:06:53.355972+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_clean_scrub_stamp":"2024-09-11T08:06:44.606510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:51:52.439955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.923048+0000","last_change":"2024-09-11T08:06:39.608517+0000","last_active":"2024-09-11T08:06:53.923048+0000","last_peered":"2024-09-11T08:06:53.923048+0000","last_clean":"2024-09-11T08:06:53.923048+0000","last_became_active":"2024-09-11T08:06:39.608077+0000","last_became_peered":"2024-09-11T08:06:39.608077+0000","last_unstale":"2024-09-11T08:06:53.923048+0000","last_undegraded":"2024-09-11T08:06:53.923048+0000","last_fullsized":"2024-09-11T08:06:53.923048+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_clean_scrub_stamp":"2024-09-11T08:06:38.581279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:43:30.293096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"59'5","reported_seq":26,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.356012+0000","last_change":"2024-09-11T08:06:41.620396+0000","last_active":"2024-09-11T08:06:53.356012+0000","last_peered":"2024-09-11T08:06:53.356012+0000","last_clean":"2024-09-11T08:06:53.356012+0000","last_became_active":"2024-09-11T08:06:41.619929+0000","last_became_peered":"2024-09-11T08:06:41.619929+0000","last_unstale":"2024-09-11T08:06:53.356012+0000","last_undegraded":"2024-09-11T08:06:53.356012+0000","last_fullsized":"2024-09-11T08:06:53.356012+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_clean_scrub_stamp":"2024-09-11T08:06:40.591817+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:10:55.772088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.922331+0000","last_change":"2024-09-11T08:06:43.624549+0000","last_active":"2024-09-11T08:06:53.922331+0000","last_peered":"2024-09-11T08:06:53.922331+0000","last_clean":"2024-09-11T08:06:53.922331+0000","last_became_active":"2024-09-11T08:06:43.624269+0000","last_became_peered":"2024-09-11T08:06:43.624269+0000","last_unstale":"2024-09-11T08:06:53.922331+0000","last_undegraded":"2024-09-11T08:06:53.922331+0000","last_fullsized":"2024-09-11T08:06:53.922331+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_clean_scrub_stamp":"2024-09-11T08:06:42.602559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:46.845168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.921614+0000","last_change":"2024-09-11T08:06:45.626788+0000","last_active":"2024-09-11T08:06:53.921614+0000","last_peered":"2024-09-11T08:06:53.921614+0000","last_clean":"2024-09-11T08:06:53.921614+0000","last_became_active":"2024-09-11T08:06:45.626296+0000","last_became_peered":"2024-09-11T08:06:45.626296+0000","last_unstale":"2024-09-11T08:06:53.921614+0000","last_undegraded":"2024-09-11T08:06:53.921614+0000","last_fullsized":"2024-09-11T08:06:53.921614+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_clean_scrub_stamp":"2024-09-11T08:06:44.606510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:18:33.095602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.922675+0000","last_change":"2024-09-11T08:06:39.602077+0000","last_active":"2024-09-11T08:06:53.922675+0000","last_peered":"2024-09-11T08:06:53.922675+0000","last_clean":"2024-09-11T08:06:53.922675+0000","last_became_active":"2024-09-11T08:06:39.601816+0000","last_became_peered":"2024-09-11T08:06:39.601816+0000","last_unstale":"2024-09-11T08:06:53.922675+0000","last_undegraded":"2024-09-11T08:06:53.922675+0000","last_fullsized":"2024-09-11T08:06:53.922675+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_clean_scrub_stamp":"2024-09-11T08:06:38.581279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:22:17.763220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"59'9","reported_seq":32,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.356154+0000","last_change":"2024-09-11T08:06:41.613120+0000","last_active":"2024-09-11T08:06:53.356154+0000","last_peered":"2024-09-11T08:06:53.356154+0000","last_clean":"2024-09-11T08:06:53.356154+0000","last_became_active":"2024-09-11T08:06:41.612829+0000","last_became_peered":"2024-09-11T08:06:41.612829+0000","last_unstale":"2024-09-11T08:06:53.356154+0000","last_undegraded":"2024-09-11T08:06:53.356154+0000","last_fullsized":"2024-09-11T08:06:53.356154+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_clean_scrub_stamp":"2024-09-11T08:06:40.591817+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:18:05.795092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.923237+0000","last_change":"2024-09-11T08:06:43.627062+0000","last_active":"2024-09-11T08:06:53.923237+0000","last_peered":"2024-09-11T08:06:53.923237+0000","last_clean":"2024-09-11T08:06:53.923237+0000","last_became_active":"2024-09-11T08:06:43.626763+0000","last_became_peered":"2024-09-11T08:06:43.626763+0000","last_unstale":"2024-09-11T08:06:53.923237+0000","last_undegraded":"2024-09-11T08:06:53.923237+0000","last_fullsized":"2024-09-11T08:06:53.923237+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_clean_scrub_stamp":"2024-09-11T08:06:42.602559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:19:00.953811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.357093+0000","last_change":"2024-09-11T08:06:45.627878+0000","last_active":"2024-09-11T08:06:53.357093+0000","last_peered":"2024-09-11T08:06:53.357093+0000","last_clean":"2024-09-11T08:06:53.357093+0000","last_became_active":"2024-09-11T08:06:45.627608+0000","last_became_peered":"2024-09-11T08:06:45.627608+0000","last_unstale":"2024-09-11T08:06:53.357093+0000","last_undegraded":"2024-09-11T08:06:53.357093+0000","last_fullsized":"2024-09-11T08:06:53.357093+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_clean_scrub_stamp":"2024-09-11T08:06:44.606510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:40:58.490513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.d","version":"59'17","reported_seq":44,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.356350+0000","last_change":"2024-09-11T08:06:41.620451+0000","last_active":"2024-09-11T08:06:53.356350+0000","last_peered":"2024-09-11T08:06:53.356350+0000","last_clean":"2024-09-11T08:06:53.356350+0000","last_became_active":"2024-09-11T08:06:41.620134+0000","last_became_peered":"2024-09-11T08:06:41.620134+0000","last_unstale":"2024-09-11T08:06:53.356350+0000","last_undegraded":"2024-09-11T08:06:53.356350+0000","last_fullsized":"2024-09-11T08:06:53.356350+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_clean_scrub_stamp":"2024-09-11T08:06:40.591817+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:18:41.188706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.922343+0000","last_change":"2024-09-11T08:06:39.603012+0000","last_active":"2024-09-11T08:06:53.922343+0000","last_peered":"2024-09-11T08:06:53.922343+0000","last_clean":"2024-09-11T08:06:53.922343+0000","last_became_active":"2024-09-11T08:06:39.602726+0000","last_became_peered":"2024-09-11T08:06:39.602726+0000","last_unstale":"2024-09-11T08:06:53.922343+0000","last_undegraded":"2024-09-11T08:06:53.922343+0000","last_fullsized":"2024-09-11T08:06:53.922343+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_clean_scrub_stamp":"2024-09-11T08:06:38.581279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:33:33.441740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.922915+0000","last_change":"2024-09-11T08:06:43.626619+0000","last_active":"2024-09-11T08:06:53.922915+0000","last_peered":"2024-09-11T08:06:53.922915+0000","last_clean":"2024-09-11T08:06:53.922915+0000","last_became_active":"2024-09-11T08:06:43.626348+0000","last_became_peered":"2024-09-11T08:06:43.626348+0000","last_unstale":"2024-09-11T08:06:53.922915+0000","last_undegraded":"2024-09-11T08:06:53.922915+0000","last_fullsized":"2024-09-11T08:06:53.922915+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_clean_scrub_stamp":"2024-09-11T08:06:42.602559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:13:51.077967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.358220+0000","last_change":"2024-09-11T08:06:45.624937+0000","last_active":"2024-09-11T08:06:53.358220+0000","last_peered":"2024-09-11T08:06:53.358220+0000","last_clean":"2024-09-11T08:06:53.358220+0000","last_became_active":"2024-09-11T08:06:45.624654+0000","last_became_peered":"2024-09-11T08:06:45.624654+0000","last_unstale":"2024-09-11T08:06:53.358220+0000","last_undegraded":"2024-09-11T08:06:53.358220+0000","last_fullsized":"2024-09-11T08:06:53.358220+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_clean_scrub_stamp":"2024-09-11T08:06:44.606510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:52:35.974581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"59'10","reported_seq":31,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.356179+0000","last_change":"2024-09-11T08:06:41.612868+0000","last_active":"2024-09-11T08:06:53.356179+0000","last_peered":"2024-09-11T08:06:53.356179+0000","last_clean":"2024-09-11T08:06:53.356179+0000","last_became_active":"2024-09-11T08:06:41.612541+0000","last_became_peered":"2024-09-11T08:06:41.612541+0000","last_unstale":"2024-09-11T08:06:53.356179+0000","last_undegraded":"2024-09-11T08:06:53.356179+0000","last_fullsized":"2024-09-11T08:06:53.356179+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_clean_scrub_stamp":"2024-09-11T08:06:40.591817+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:04:23.460925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.923245+0000","last_change":"2024-09-11T08:06:39.605945+0000","last_active":"2024-09-11T08:06:53.923245+0000","last_peered":"2024-09-11T08:06:53.923245+0000","last_clean":"2024-09-11T08:06:53.923245+0000","last_became_active":"2024-09-11T08:06:39.605672+0000","last_became_peered":"2024-09-11T08:06:39.605672+0000","last_unstale":"2024-09-11T08:06:53.923245+0000","last_undegraded":"2024-09-11T08:06:53.923245+0000","last_fullsized":"2024-09-11T08:06:53.923245+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_clean_scrub_stamp":"2024-09-11T08:06:38.581279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:35:07.469717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"59'8","reported_seq":38,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:07:53.122866+0000","last_change":"2024-09-11T08:06:43.628373+0000","last_active":"2024-09-11T08:07:53.122866+0000","last_peered":"2024-09-11T08:07:53.122866+0000","last_clean":"2024-09-11T08:07:53.122866+0000","last_became_active":"2024-09-11T08:06:43.628087+0000","last_became_peered":"2024-09-11T08:06:43.628087+0000","last_unstale":"2024-09-11T08:07:53.122866+0000","last_undegraded":"2024-09-11T08:07:53.122866+0000","last_fullsized":"2024-09-11T08:07:53.122866+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_clean_scrub_stamp":"2024-09-11T08:06:42.602559+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:47:05.748888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.356309+0000","last_change":"2024-09-11T08:06:45.627617+0000","last_active":"2024-09-11T08:06:53.356309+0000","last_peered":"2024-09-11T08:06:53.356309+0000","last_clean":"2024-09-11T08:06:53.356309+0000","last_became_active":"2024-09-11T08:06:45.626825+0000","last_became_peered":"2024-09-11T08:06:45.626825+0000","last_unstale":"2024-09-11T08:06:53.356309+0000","last_undegraded":"2024-09-11T08:06:53.356309+0000","last_fullsized":"2024-09-11T08:06:53.356309+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_clean_scrub_stamp":"2024-09-11T08:06:44.606510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:02:05.584271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"59'15","reported_seq":41,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.923558+0000","last_change":"2024-09-11T08:06:41.619820+0000","last_active":"2024-09-11T08:06:53.923558+0000","last_peered":"2024-09-11T08:06:53.923558+0000","last_clean":"2024-09-11T08:06:53.923558+0000","last_became_active":"2024-09-11T08:06:41.619534+0000","last_became_peered":"2024-09-11T08:06:41.619534+0000","last_unstale":"2024-09-11T08:06:53.923558+0000","last_undegraded":"2024-09-11T08:06:53.923558+0000","last_fullsized":"2024-09-11T08:06:53.923558+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_clean_scrub_stamp":"2024-09-11T08:06:40.591817+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:40:30.644923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.922840+0000","last_change":"2024-09-11T08:06:39.608392+0000","last_active":"2024-09-11T08:06:53.922840+0000","last_peered":"2024-09-11T08:06:53.922840+0000","last_clean":"2024-09-11T08:06:53.922840+0000","last_became_active":"2024-09-11T08:06:39.608060+0000","last_became_peered":"2024-09-11T08:06:39.608060+0000","last_unstale":"2024-09-11T08:06:53.922840+0000","last_undegraded":"2024-09-11T08:06:53.922840+0000","last_fullsized":"2024-09-11T08:06:53.922840+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_clean_scrub_stamp":"2024-09-11T08:06:38.581279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:14:47.084168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"59'8","reported_seq":38,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:07:53.123125+0000","last_change":"2024-09-11T08:06:43.629899+0000","last_active":"2024-09-11T08:07:53.123125+0000","last_peered":"2024-09-11T08:07:53.123125+0000","last_clean":"2024-09-11T08:07:53.123125+0000","last_became_active":"2024-09-11T08:06:43.629636+0000","last_became_peered":"2024-09-11T08:06:43.629636+0000","last_unstale":"2024-09-11T08:07:53.123125+0000","last_undegraded":"2024-09-11T08:07:53.123125+0000","last_fullsized":"2024-09-11T08:07:53.123125+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_clean_scrub_stamp":"2024-09-11T08:06:42.602559+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:23:53.155784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.923667+0000","last_change":"2024-09-11T08:06:45.632863+0000","last_active":"2024-09-11T08:06:53.923667+0000","last_peered":"2024-09-11T08:06:53.923667+0000","last_clean":"2024-09-11T08:06:53.923667+0000","last_became_active":"2024-09-11T08:06:45.632518+0000","last_became_peered":"2024-09-11T08:06:45.632518+0000","last_unstale":"2024-09-11T08:06:53.923667+0000","last_undegraded":"2024-09-11T08:06:53.923667+0000","last_fullsized":"2024-09-11T08:06:53.923667+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_clean_scrub_stamp":"2024-09-11T08:06:44.606510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:50:43.840099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"59'11","reported_seq":35,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.356459+0000","last_change":"2024-09-11T08:06:41.620101+0000","last_active":"2024-09-11T08:06:53.356459+0000","last_peered":"2024-09-11T08:06:53.356459+0000","last_clean":"2024-09-11T08:06:53.356459+0000","last_became_active":"2024-09-11T08:06:41.619521+0000","last_became_peered":"2024-09-11T08:06:41.619521+0000","last_unstale":"2024-09-11T08:06:53.356459+0000","last_undegraded":"2024-09-11T08:06:53.356459+0000","last_fullsized":"2024-09-11T08:06:53.356459+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_clean_scrub_stamp":"2024-09-11T08:06:40.591817+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:12:28.192461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.922976+0000","last_change":"2024-09-11T08:06:39.609950+0000","last_active":"2024-09-11T08:06:53.922976+0000","last_peered":"2024-09-11T08:06:53.922976+0000","last_clean":"2024-09-11T08:06:53.922976+0000","last_became_active":"2024-09-11T08:06:39.609215+0000","last_became_peered":"2024-09-11T08:06:39.609215+0000","last_unstale":"2024-09-11T08:06:53.922976+0000","last_undegraded":"2024-09-11T08:06:53.922976+0000","last_fullsized":"2024-09-11T08:06:53.922976+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_clean_scrub_stamp":"2024-09-11T08:06:38.581279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:07:31.708252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.923809+0000","last_change":"2024-09-11T08:06:43.627822+0000","last_active":"2024-09-11T08:06:53.923809+0000","last_peered":"2024-09-11T08:06:53.923809+0000","last_clean":"2024-09-11T08:06:53.923809+0000","last_became_active":"2024-09-11T08:06:43.627558+0000","last_became_peered":"2024-09-11T08:06:43.627558+0000","last_unstale":"2024-09-11T08:06:53.923809+0000","last_undegraded":"2024-09-11T08:06:53.923809+0000","last_fullsized":"2024-09-11T08:06:53.923809+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_clean_scrub_stamp":"2024-09-11T08:06:42.602559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:50:09.567644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.505178+0000","last_change":"2024-09-11T08:06:46.172869+0000","last_active":"2024-09-11T08:06:53.505178+0000","last_peered":"2024-09-11T08:06:53.505178+0000","last_clean":"2024-09-11T08:06:53.505178+0000","last_became_active":"2024-09-11T08:06:46.172458+0000","last_became_peered":"2024-09-11T08:06:46.172458+0000","last_unstale":"2024-09-11T08:06:53.505178+0000","last_undegraded":"2024-09-11T08:06:53.505178+0000","last_fullsized":"2024-09-11T08:06:53.505178+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_clean_scrub_stamp":"2024-09-11T08:06:44.606510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:10:31.229111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"59'14","reported_seq":37,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.356071+0000","last_change":"2024-09-11T08:06:41.613763+0000","last_active":"2024-09-11T08:06:53.356071+0000","last_peered":"2024-09-11T08:06:53.356071+0000","last_clean":"2024-09-11T08:06:53.356071+0000","last_became_active":"2024-09-11T08:06:41.613474+0000","last_became_peered":"2024-09-11T08:06:41.613474+0000","last_unstale":"2024-09-11T08:06:53.356071+0000","last_undegraded":"2024-09-11T08:06:53.356071+0000","last_fullsized":"2024-09-11T08:06:53.356071+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_clean_scrub_stamp":"2024-09-11T08:06:40.591817+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:08:59.356386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.922931+0000","last_change":"2024-09-11T08:06:39.603231+0000","last_active":"2024-09-11T08:06:53.922931+0000","last_peered":"2024-09-11T08:06:53.922931+0000","last_clean":"2024-09-11T08:06:53.922931+0000","last_became_active":"2024-09-11T08:06:39.602503+0000","last_became_peered":"2024-09-11T08:06:39.602503+0000","last_unstale":"2024-09-11T08:06:53.922931+0000","last_undegraded":"2024-09-11T08:06:53.922931+0000","last_fullsized":"2024-09-11T08:06:53.922931+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_clean_scrub_stamp":"2024-09-11T08:06:38.581279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:16:47.034701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"59'8","reported_seq":41,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:07:53.122923+0000","last_change":"2024-09-11T08:06:43.627128+0000","last_active":"2024-09-11T08:07:53.122923+0000","last_peered":"2024-09-11T08:07:53.122923+0000","last_clean":"2024-09-11T08:07:53.122923+0000","last_became_active":"2024-09-11T08:06:43.626849+0000","last_became_peered":"2024-09-11T08:06:43.626849+0000","last_unstale":"2024-09-11T08:07:53.122923+0000","last_undegraded":"2024-09-11T08:07:53.122923+0000","last_fullsized":"2024-09-11T08:07:53.122923+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_clean_scrub_stamp":"2024-09-11T08:06:42.602559+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:24:27.021095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.922203+0000","last_change":"2024-09-11T08:06:46.175440+0000","last_active":"2024-09-11T08:06:53.922203+0000","last_peered":"2024-09-11T08:06:53.922203+0000","last_clean":"2024-09-11T08:06:53.922203+0000","last_became_active":"2024-09-11T08:06:46.175157+0000","last_became_peered":"2024-09-11T08:06:46.175157+0000","last_unstale":"2024-09-11T08:06:53.922203+0000","last_undegraded":"2024-09-11T08:06:53.922203+0000","last_fullsized":"2024-09-11T08:06:53.922203+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_clean_scrub_stamp":"2024-09-11T08:06:44.606510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:49:16.423291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"59'18","reported_seq":48,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.504760+0000","last_change":"2024-09-11T08:06:42.465901+0000","last_active":"2024-09-11T08:06:53.504760+0000","last_peered":"2024-09-11T08:06:53.504760+0000","last_clean":"2024-09-11T08:06:53.504760+0000","last_became_active":"2024-09-11T08:06:42.465624+0000","last_became_peered":"2024-09-11T08:06:42.465624+0000","last_unstale":"2024-09-11T08:06:53.504760+0000","last_undegraded":"2024-09-11T08:06:53.504760+0000","last_fullsized":"2024-09-11T08:06:53.504760+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_clean_scrub_stamp":"2024-09-11T08:06:40.591817+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:37:31.078992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.923650+0000","last_change":"2024-09-11T08:06:39.610426+0000","last_active":"2024-09-11T08:06:53.923650+0000","last_peered":"2024-09-11T08:06:53.923650+0000","last_clean":"2024-09-11T08:06:53.923650+0000","last_became_active":"2024-09-11T08:06:39.610142+0000","last_became_peered":"2024-09-11T08:06:39.610142+0000","last_unstale":"2024-09-11T08:06:53.923650+0000","last_undegraded":"2024-09-11T08:06:53.923650+0000","last_fullsized":"2024-09-11T08:06:53.923650+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_clean_scrub_stamp":"2024-09-11T08:06:38.581279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:00.085742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.355863+0000","last_change":"2024-09-11T08:06:43.621918+0000","last_active":"2024-09-11T08:06:53.355863+0000","last_peered":"2024-09-11T08:06:53.355863+0000","last_clean":"2024-09-11T08:06:53.355863+0000","last_became_active":"2024-09-11T08:06:43.620966+0000","last_became_peered":"2024-09-11T08:06:43.620966+0000","last_unstale":"2024-09-11T08:06:53.355863+0000","last_undegraded":"2024-09-11T08:06:53.355863+0000","last_fullsized":"2024-09-11T08:06:53.355863+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_clean_scrub_stamp":"2024-09-11T08:06:42.602559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:28:54.651591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.356153+0000","last_change":"2024-09-11T08:06:45.628163+0000","last_active":"2024-09-11T08:06:53.356153+0000","last_peered":"2024-09-11T08:06:53.356153+0000","last_clean":"2024-09-11T08:06:53.356153+0000","last_became_active":"2024-09-11T08:06:45.627725+0000","last_became_peered":"2024-09-11T08:06:45.627725+0000","last_unstale":"2024-09-11T08:06:53.356153+0000","last_undegraded":"2024-09-11T08:06:53.356153+0000","last_fullsized":"2024-09-11T08:06:53.356153+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_clean_scrub_stamp":"2024-09-11T08:06:44.606510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:14:54.778445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"59'19","reported_seq":52,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.357887+0000","last_change":"2024-09-11T08:06:42.463317+0000","last_active":"2024-09-11T08:06:53.357887+0000","last_peered":"2024-09-11T08:06:53.357887+0000","last_clean":"2024-09-11T08:06:53.357887+0000","last_became_active":"2024-09-11T08:06:42.462993+0000","last_became_peered":"2024-09-11T08:06:42.462993+0000","last_unstale":"2024-09-11T08:06:53.357887+0000","last_undegraded":"2024-09-11T08:06:53.357887+0000","last_fullsized":"2024-09-11T08:06:53.357887+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_clean_scrub_stamp":"2024-09-11T08:06:40.591817+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:21:04.443228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.923073+0000","last_change":"2024-09-11T08:06:39.608340+0000","last_active":"2024-09-11T08:06:53.923073+0000","last_peered":"2024-09-11T08:06:53.923073+0000","last_clean":"2024-09-11T08:06:53.923073+0000","last_became_active":"2024-09-11T08:06:39.607713+0000","last_became_peered":"2024-09-11T08:06:39.607713+0000","last_unstale":"2024-09-11T08:06:53.923073+0000","last_undegraded":"2024-09-11T08:06:53.923073+0000","last_fullsized":"2024-09-11T08:06:53.923073+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_clean_scrub_stamp":"2024-09-11T08:06:38.581279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:53:46.241083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.922610+0000","last_change":"2024-09-11T08:06:43.627376+0000","last_active":"2024-09-11T08:06:53.922610+0000","last_peered":"2024-09-11T08:06:53.922610+0000","last_clean":"2024-09-11T08:06:53.922610+0000","last_became_active":"2024-09-11T08:06:43.627099+0000","last_became_peered":"2024-09-11T08:06:43.627099+0000","last_unstale":"2024-09-11T08:06:53.922610+0000","last_undegraded":"2024-09-11T08:06:53.922610+0000","last_fullsized":"2024-09-11T08:06:53.922610+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_clean_scrub_stamp":"2024-09-11T08:06:42.602559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:06:03.180648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.922471+0000","last_change":"2024-09-11T08:06:46.172840+0000","last_active":"2024-09-11T08:06:53.922471+0000","last_peered":"2024-09-11T08:06:53.922471+0000","last_clean":"2024-09-11T08:06:53.922471+0000","last_became_active":"2024-09-11T08:06:46.172430+0000","last_became_peered":"2024-09-11T08:06:46.172430+0000","last_unstale":"2024-09-11T08:06:53.922471+0000","last_undegraded":"2024-09-11T08:06:53.922471+0000","last_fullsized":"2024-09-11T08:06:53.922471+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_clean_scrub_stamp":"2024-09-11T08:06:44.606510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:46:46.231346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"59'10","reported_seq":31,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.923176+0000","last_change":"2024-09-11T08:06:41.618705+0000","last_active":"2024-09-11T08:06:53.923176+0000","last_peered":"2024-09-11T08:06:53.923176+0000","last_clean":"2024-09-11T08:06:53.923176+0000","last_became_active":"2024-09-11T08:06:41.617495+0000","last_became_peered":"2024-09-11T08:06:41.617495+0000","last_unstale":"2024-09-11T08:06:53.923176+0000","last_undegraded":"2024-09-11T08:06:53.923176+0000","last_fullsized":"2024-09-11T08:06:53.923176+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_clean_scrub_stamp":"2024-09-11T08:06:40.591817+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:24:32.460028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.923109+0000","last_change":"2024-09-11T08:06:39.606512+0000","last_active":"2024-09-11T08:06:53.923109+0000","last_peered":"2024-09-11T08:06:53.923109+0000","last_clean":"2024-09-11T08:06:53.923109+0000","last_became_active":"2024-09-11T08:06:39.606234+0000","last_became_peered":"2024-09-11T08:06:39.606234+0000","last_unstale":"2024-09-11T08:06:53.923109+0000","last_undegraded":"2024-09-11T08:06:53.923109+0000","last_fullsized":"2024-09-11T08:06:53.923109+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_clean_scrub_stamp":"2024-09-11T08:06:38.581279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:35:51.447680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"59'8","reported_seq":38,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:07:53.122872+0000","last_change":"2024-09-11T08:06:43.630234+0000","last_active":"2024-09-11T08:07:53.122872+0000","last_peered":"2024-09-11T08:07:53.122872+0000","last_clean":"2024-09-11T08:07:53.122872+0000","last_became_active":"2024-09-11T08:06:43.629808+0000","last_became_peered":"2024-09-11T08:06:43.629808+0000","last_unstale":"2024-09-11T08:07:53.122872+0000","last_undegraded":"2024-09-11T08:07:53.122872+0000","last_fullsized":"2024-09-11T08:07:53.122872+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_clean_scrub_stamp":"2024-09-11T08:06:42.602559+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:37:08.879992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.356941+0000","last_change":"2024-09-11T08:06:45.624297+0000","last_active":"2024-09-11T08:06:53.356941+0000","last_peered":"2024-09-11T08:06:53.356941+0000","last_clean":"2024-09-11T08:06:53.356941+0000","last_became_active":"2024-09-11T08:06:45.624005+0000","last_became_peered":"2024-09-11T08:06:45.624005+0000","last_unstale":"2024-09-11T08:06:53.356941+0000","last_undegraded":"2024-09-11T08:06:53.356941+0000","last_fullsized":"2024-09-11T08:06:53.356941+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_clean_scrub_stamp":"2024-09-11T08:06:44.606510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:27:21.190292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"59'28","reported_seq":80,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:07:53.122849+0000","last_change":"2024-09-11T08:06:41.622452+0000","last_active":"2024-09-11T08:07:53.122849+0000","last_peered":"2024-09-11T08:07:53.122849+0000","last_clean":"2024-09-11T08:07:53.122849+0000","last_became_active":"2024-09-11T08:06:41.622118+0000","last_became_peered":"2024-09-11T08:06:41.622118+0000","last_unstale":"2024-09-11T08:07:53.122849+0000","last_undegraded":"2024-09-11T08:07:53.122849+0000","last_fullsized":"2024-09-11T08:07:53.122849+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_clean_scrub_stamp":"2024-09-11T08:06:40.591817+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:15:46.142935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.923538+0000","last_change":"2024-09-11T08:06:39.604374+0000","last_active":"2024-09-11T08:06:53.923538+0000","last_peered":"2024-09-11T08:06:53.923538+0000","last_clean":"2024-09-11T08:06:53.923538+0000","last_became_active":"2024-09-11T08:06:39.604076+0000","last_became_peered":"2024-09-11T08:06:39.604076+0000","last_unstale":"2024-09-11T08:06:53.923538+0000","last_undegraded":"2024-09-11T08:06:53.923538+0000","last_fullsized":"2024-09-11T08:06:53.923538+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_clean_scrub_stamp":"2024-09-11T08:06:38.581279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:05:34.368960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.357382+0000","last_change":"2024-09-11T08:06:43.619964+0000","last_active":"2024-09-11T08:06:53.357382+0000","last_peered":"2024-09-11T08:06:53.357382+0000","last_clean":"2024-09-11T08:06:53.357382+0000","last_became_active":"2024-09-11T08:06:43.619685+0000","last_became_peered":"2024-09-11T08:06:43.619685+0000","last_unstale":"2024-09-11T08:06:53.357382+0000","last_undegraded":"2024-09-11T08:06:53.357382+0000","last_fullsized":"2024-09-11T08:06:53.357382+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_clean_scrub_stamp":"2024-09-11T08:06:42.602559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:14:41.595586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.505236+0000","last_change":"2024-09-11T08:06:45.627245+0000","last_active":"2024-09-11T08:06:53.505236+0000","last_peered":"2024-09-11T08:06:53.505236+0000","last_clean":"2024-09-11T08:06:53.505236+0000","last_became_active":"2024-09-11T08:06:45.626919+0000","last_became_peered":"2024-09-11T08:06:45.626919+0000","last_unstale":"2024-09-11T08:06:53.505236+0000","last_undegraded":"2024-09-11T08:06:53.505236+0000","last_fullsized":"2024-09-11T08:06:53.505236+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_clean_scrub_stamp":"2024-09-11T08:06:44.606510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:59:15.307403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"59'13","reported_seq":43,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.923622+0000","last_change":"2024-09-11T08:06:41.618883+0000","last_active":"2024-09-11T08:06:53.923622+0000","last_peered":"2024-09-11T08:06:53.923622+0000","last_clean":"2024-09-11T08:06:53.923622+0000","last_became_active":"2024-09-11T08:06:41.618616+0000","last_became_peered":"2024-09-11T08:06:41.618616+0000","last_unstale":"2024-09-11T08:06:53.923622+0000","last_undegraded":"2024-09-11T08:06:53.923622+0000","last_fullsized":"2024-09-11T08:06:53.923622+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_clean_scrub_stamp":"2024-09-11T08:06:40.591817+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:09:02.784263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"59'5","reported_seq":103,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:08:02.044505+0000","last_change":"2024-09-11T08:06:43.619846+0000","last_active":"2024-09-11T08:08:02.044505+0000","last_peered":"2024-09-11T08:08:02.044505+0000","last_clean":"2024-09-11T08:08:02.044505+0000","last_became_active":"2024-09-11T08:06:39.609471+0000","last_became_peered":"2024-09-11T08:06:39.609471+0000","last_unstale":"2024-09-11T08:08:02.044505+0000","last_undegraded":"2024-09-11T08:08:02.044505+0000","last_fullsized":"2024-09-11T08:08:02.044505+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_clean_scrub_stamp":"2024-09-11T08:06:38.581279+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:00:36.163114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056896900000000001,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":77,"num_read_kb":72,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.357000+0000","last_change":"2024-09-11T08:06:39.606585+0000","last_active":"2024-09-11T08:06:53.357000+0000","last_peered":"2024-09-11T08:06:53.357000+0000","last_clean":"2024-09-11T08:06:53.357000+0000","last_became_active":"2024-09-11T08:06:39.606331+0000","last_became_peered":"2024-09-11T08:06:39.606331+0000","last_unstale":"2024-09-11T08:06:53.357000+0000","last_undegraded":"2024-09-11T08:06:53.357000+0000","last_fullsized":"2024-09-11T08:06:53.357000+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_clean_scrub_stamp":"2024-09-11T08:06:38.581279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:21:21.466087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.357889+0000","last_change":"2024-09-11T08:06:43.629359+0000","last_active":"2024-09-11T08:06:53.357889+0000","last_peered":"2024-09-11T08:06:53.357889+0000","last_clean":"2024-09-11T08:06:53.357889+0000","last_became_active":"2024-09-11T08:06:43.629097+0000","last_became_peered":"2024-09-11T08:06:43.629097+0000","last_unstale":"2024-09-11T08:06:53.357889+0000","last_undegraded":"2024-09-11T08:06:53.357889+0000","last_fullsized":"2024-09-11T08:06:53.357889+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_clean_scrub_stamp":"2024-09-11T08:06:42.602559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:22:55.100376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.922546+0000","last_change":"2024-09-11T08:06:46.175308+0000","last_active":"2024-09-11T08:06:53.922546+0000","last_peered":"2024-09-11T08:06:53.922546+0000","last_clean":"2024-09-11T08:06:53.922546+0000","last_became_active":"2024-09-11T08:06:46.175011+0000","last_became_peered":"2024-09-11T08:06:46.175011+0000","last_unstale":"2024-09-11T08:06:53.922546+0000","last_undegraded":"2024-09-11T08:06:53.922546+0000","last_fullsized":"2024-09-11T08:06:53.922546+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_clean_scrub_stamp":"2024-09-11T08:06:44.606510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:00:43.461643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"59'12","reported_seq":34,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.358562+0000","last_change":"2024-09-11T08:06:41.616886+0000","last_active":"2024-09-11T08:06:53.358562+0000","last_peered":"2024-09-11T08:06:53.358562+0000","last_clean":"2024-09-11T08:06:53.358562+0000","last_became_active":"2024-09-11T08:06:41.616634+0000","last_became_peered":"2024-09-11T08:06:41.616634+0000","last_unstale":"2024-09-11T08:06:53.358562+0000","last_undegraded":"2024-09-11T08:06:53.358562+0000","last_fullsized":"2024-09-11T08:06:53.358562+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_clean_scrub_stamp":"2024-09-11T08:06:40.591817+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:52:19.243060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"54'1","reported_seq":24,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.358142+0000","last_change":"2024-09-11T08:06:43.610415+0000","last_active":"2024-09-11T08:06:53.358142+0000","last_peered":"2024-09-11T08:06:53.358142+0000","last_clean":"2024-09-11T08:06:53.358142+0000","last_became_active":"2024-09-11T08:06:39.607546+0000","last_became_peered":"2024-09-11T08:06:39.607546+0000","last_unstale":"2024-09-11T08:06:53.358142+0000","last_undegraded":"2024-09-11T08:06:53.358142+0000","last_fullsized":"2024-09-11T08:06:53.358142+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_clean_scrub_stamp":"2024-09-11T08:06:38.581279+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:47:48.094686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000423239,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"52'1","reported_seq":25,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.923066+0000","last_change":"2024-09-11T08:06:39.609049+0000","last_active":"2024-09-11T08:06:53.923066+0000","last_peered":"2024-09-11T08:06:53.923066+0000","last_clean":"2024-09-11T08:06:53.923066+0000","last_became_active":"2024-09-11T08:06:39.608473+0000","last_became_peered":"2024-09-11T08:06:39.608473+0000","last_unstale":"2024-09-11T08:06:53.923066+0000","last_undegraded":"2024-09-11T08:06:53.923066+0000","last_fullsized":"2024-09-11T08:06:53.923066+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_clean_scrub_stamp":"2024-09-11T08:06:38.581279+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:47:06.604637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.923877+0000","last_change":"2024-09-11T08:06:43.627680+0000","last_active":"2024-09-11T08:06:53.923877+0000","last_peered":"2024-09-11T08:06:53.923877+0000","last_clean":"2024-09-11T08:06:53.923877+0000","last_became_active":"2024-09-11T08:06:43.627405+0000","last_became_peered":"2024-09-11T08:06:43.627405+0000","last_unstale":"2024-09-11T08:06:53.923877+0000","last_undegraded":"2024-09-11T08:06:53.923877+0000","last_fullsized":"2024-09-11T08:06:53.923877+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_clean_scrub_stamp":"2024-09-11T08:06:42.602559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:56:12.256792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.922327+0000","last_change":"2024-09-11T08:06:45.626461+0000","last_active":"2024-09-11T08:06:53.922327+0000","last_peered":"2024-09-11T08:06:53.922327+0000","last_clean":"2024-09-11T08:06:53.922327+0000","last_became_active":"2024-09-11T08:06:45.626124+0000","last_became_peered":"2024-09-11T08:06:45.626124+0000","last_unstale":"2024-09-11T08:06:53.922327+0000","last_undegraded":"2024-09-11T08:06:53.922327+0000","last_fullsized":"2024-09-11T08:06:53.922327+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_clean_scrub_stamp":"2024-09-11T08:06:44.606510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:38:52.071915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"59'14","reported_seq":38,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.922489+0000","last_change":"2024-09-11T08:06:42.464026+0000","last_active":"2024-09-11T08:06:53.922489+0000","last_peered":"2024-09-11T08:06:53.922489+0000","last_clean":"2024-09-11T08:06:53.922489+0000","last_became_active":"2024-09-11T08:06:42.463593+0000","last_became_peered":"2024-09-11T08:06:42.463593+0000","last_unstale":"2024-09-11T08:06:53.922489+0000","last_undegraded":"2024-09-11T08:06:53.922489+0000","last_fullsized":"2024-09-11T08:06:53.922489+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_clean_scrub_stamp":"2024-09-11T08:06:40.591817+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:56:48.344165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"56'2","reported_seq":25,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.505088+0000","last_change":"2024-09-11T08:06:43.617993+0000","last_active":"2024-09-11T08:06:53.505088+0000","last_peered":"2024-09-11T08:06:53.505088+0000","last_clean":"2024-09-11T08:06:53.505088+0000","last_became_active":"2024-09-11T08:06:39.602072+0000","last_became_peered":"2024-09-11T08:06:39.602072+0000","last_unstale":"2024-09-11T08:06:53.505088+0000","last_undegraded":"2024-09-11T08:06:53.505088+0000","last_fullsized":"2024-09-11T08:06:53.505088+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_clean_scrub_stamp":"2024-09-11T08:06:38.581279+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:23:58.029145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001549904,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.923605+0000","last_change":"2024-09-11T08:06:39.605901+0000","last_active":"2024-09-11T08:06:53.923605+0000","last_peered":"2024-09-11T08:06:53.923605+0000","last_clean":"2024-09-11T08:06:53.923605+0000","last_became_active":"2024-09-11T08:06:39.605647+0000","last_became_peered":"2024-09-11T08:06:39.605647+0000","last_unstale":"2024-09-11T08:06:53.923605+0000","last_undegraded":"2024-09-11T08:06:53.923605+0000","last_fullsized":"2024-09-11T08:06:53.923605+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_clean_scrub_stamp":"2024-09-11T08:06:38.581279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:32:10.154369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"61'79","reported_seq":52,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:07:19.108170+0000","last_change":"2024-09-11T08:06:29.133512+0000","last_active":"2024-09-11T08:07:19.108170+0000","last_peered":"2024-09-11T08:07:19.108170+0000","last_clean":"2024-09-11T08:07:19.108170+0000","last_became_active":"2024-09-11T08:06:29.127939+0000","last_became_peered":"2024-09-11T08:06:29.127939+0000","last_unstale":"2024-09-11T08:07:19.108170+0000","last_undegraded":"2024-09-11T08:07:19.108170+0000","last_fullsized":"2024-09-11T08:07:19.108170+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:04:29.497369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:04:29.497369+0000","last_clean_scrub_stamp":"2024-09-11T08:04:29.497369+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:30:31.230878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.922446+0000","last_change":"2024-09-11T08:06:43.628742+0000","last_active":"2024-09-11T08:06:53.922446+0000","last_peered":"2024-09-11T08:06:53.922446+0000","last_clean":"2024-09-11T08:06:53.922446+0000","last_became_active":"2024-09-11T08:06:43.628466+0000","last_became_peered":"2024-09-11T08:06:43.628466+0000","last_unstale":"2024-09-11T08:06:53.922446+0000","last_undegraded":"2024-09-11T08:06:53.922446+0000","last_fullsized":"2024-09-11T08:06:53.922446+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_clean_scrub_stamp":"2024-09-11T08:06:42.602559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:33:07.821898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.921946+0000","last_change":"2024-09-11T08:06:45.628579+0000","last_active":"2024-09-11T08:06:53.921946+0000","last_peered":"2024-09-11T08:06:53.921946+0000","last_clean":"2024-09-11T08:06:53.921946+0000","last_became_active":"2024-09-11T08:06:45.628293+0000","last_became_peered":"2024-09-11T08:06:45.628293+0000","last_unstale":"2024-09-11T08:06:53.921946+0000","last_undegraded":"2024-09-11T08:06:53.921946+0000","last_fullsized":"2024-09-11T08:06:53.921946+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_clean_scrub_stamp":"2024-09-11T08:06:44.606510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:23:51.069892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"59'19","reported_seq":47,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.922748+0000","last_change":"2024-09-11T08:06:42.464039+0000","last_active":"2024-09-11T08:06:53.922748+0000","last_peered":"2024-09-11T08:06:53.922748+0000","last_clean":"2024-09-11T08:06:53.922748+0000","last_became_active":"2024-09-11T08:06:42.462906+0000","last_became_peered":"2024-09-11T08:06:42.462906+0000","last_unstale":"2024-09-11T08:06:53.922748+0000","last_undegraded":"2024-09-11T08:06:53.922748+0000","last_fullsized":"2024-09-11T08:06:53.922748+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_clean_scrub_stamp":"2024-09-11T08:06:40.591817+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:03:12.280109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.359059+0000","last_change":"2024-09-11T08:06:39.608804+0000","last_active":"2024-09-11T08:06:53.359059+0000","last_peered":"2024-09-11T08:06:53.359059+0000","last_clean":"2024-09-11T08:06:53.359059+0000","last_became_active":"2024-09-11T08:06:39.607177+0000","last_became_peered":"2024-09-11T08:06:39.607177+0000","last_unstale":"2024-09-11T08:06:53.359059+0000","last_undegraded":"2024-09-11T08:06:53.359059+0000","last_fullsized":"2024-09-11T08:06:53.359059+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_clean_scrub_stamp":"2024-09-11T08:06:38.581279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:55:25.120756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.357344+0000","last_change":"2024-09-11T08:06:43.627101+0000","last_active":"2024-09-11T08:06:53.357344+0000","last_peered":"2024-09-11T08:06:53.357344+0000","last_clean":"2024-09-11T08:06:53.357344+0000","last_became_active":"2024-09-11T08:06:43.626809+0000","last_became_peered":"2024-09-11T08:06:43.626809+0000","last_unstale":"2024-09-11T08:06:53.357344+0000","last_undegraded":"2024-09-11T08:06:53.357344+0000","last_fullsized":"2024-09-11T08:06:53.357344+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_clean_scrub_stamp":"2024-09-11T08:06:42.602559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:48:21.116393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"59'1","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.921967+0000","last_change":"2024-09-11T08:06:45.632561+0000","last_active":"2024-09-11T08:06:53.921967+0000","last_peered":"2024-09-11T08:06:53.921967+0000","last_clean":"2024-09-11T08:06:53.921967+0000","last_became_active":"2024-09-11T08:06:45.632248+0000","last_became_peered":"2024-09-11T08:06:45.632248+0000","last_unstale":"2024-09-11T08:06:53.921967+0000","last_undegraded":"2024-09-11T08:06:53.921967+0000","last_fullsized":"2024-09-11T08:06:53.921967+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_clean_scrub_stamp":"2024-09-11T08:06:44.606510+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:36:11.933729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"59'9","reported_seq":32,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.357800+0000","last_change":"2024-09-11T08:06:41.620163+0000","last_active":"2024-09-11T08:06:53.357800+0000","last_peered":"2024-09-11T08:06:53.357800+0000","last_clean":"2024-09-11T08:06:53.357800+0000","last_became_active":"2024-09-11T08:06:41.619582+0000","last_became_peered":"2024-09-11T08:06:41.619582+0000","last_unstale":"2024-09-11T08:06:53.357800+0000","last_undegraded":"2024-09-11T08:06:53.357800+0000","last_fullsized":"2024-09-11T08:06:53.357800+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_clean_scrub_stamp":"2024-09-11T08:06:40.591817+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:46:55.251605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.923560+0000","last_change":"2024-09-11T08:06:39.606228+0000","last_active":"2024-09-11T08:06:53.923560+0000","last_peered":"2024-09-11T08:06:53.923560+0000","last_clean":"2024-09-11T08:06:53.923560+0000","last_became_active":"2024-09-11T08:06:39.605978+0000","last_became_peered":"2024-09-11T08:06:39.605978+0000","last_unstale":"2024-09-11T08:06:53.923560+0000","last_undegraded":"2024-09-11T08:06:53.923560+0000","last_fullsized":"2024-09-11T08:06:53.923560+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_clean_scrub_stamp":"2024-09-11T08:06:38.581279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:20:04.792419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.357122+0000","last_change":"2024-09-11T08:06:43.621932+0000","last_active":"2024-09-11T08:06:53.357122+0000","last_peered":"2024-09-11T08:06:53.357122+0000","last_clean":"2024-09-11T08:06:53.357122+0000","last_became_active":"2024-09-11T08:06:43.621631+0000","last_became_peered":"2024-09-11T08:06:43.621631+0000","last_unstale":"2024-09-11T08:06:53.357122+0000","last_undegraded":"2024-09-11T08:06:53.357122+0000","last_fullsized":"2024-09-11T08:06:53.357122+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_clean_scrub_stamp":"2024-09-11T08:06:42.602559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:00:18.710953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.358253+0000","last_change":"2024-09-11T08:06:45.629092+0000","last_active":"2024-09-11T08:06:53.358253+0000","last_peered":"2024-09-11T08:06:53.358253+0000","last_clean":"2024-09-11T08:06:53.358253+0000","last_became_active":"2024-09-11T08:06:45.628810+0000","last_became_peered":"2024-09-11T08:06:45.628810+0000","last_unstale":"2024-09-11T08:06:53.358253+0000","last_undegraded":"2024-09-11T08:06:53.358253+0000","last_fullsized":"2024-09-11T08:06:53.358253+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_clean_scrub_stamp":"2024-09-11T08:06:44.606510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:31:17.639950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"59'15","reported_seq":41,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.923870+0000","last_change":"2024-09-11T08:06:42.465052+0000","last_active":"2024-09-11T08:06:53.923870+0000","last_peered":"2024-09-11T08:06:53.923870+0000","last_clean":"2024-09-11T08:06:53.923870+0000","last_became_active":"2024-09-11T08:06:42.464746+0000","last_became_peered":"2024-09-11T08:06:42.464746+0000","last_unstale":"2024-09-11T08:06:53.923870+0000","last_undegraded":"2024-09-11T08:06:53.923870+0000","last_fullsized":"2024-09-11T08:06:53.923870+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_clean_scrub_stamp":"2024-09-11T08:06:40.591817+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:10:24.337047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.358070+0000","last_change":"2024-09-11T08:06:39.602562+0000","last_active":"2024-09-11T08:06:53.358070+0000","last_peered":"2024-09-11T08:06:53.358070+0000","last_clean":"2024-09-11T08:06:53.358070+0000","last_became_active":"2024-09-11T08:06:39.602202+0000","last_became_peered":"2024-09-11T08:06:39.602202+0000","last_unstale":"2024-09-11T08:06:53.358070+0000","last_undegraded":"2024-09-11T08:06:53.358070+0000","last_fullsized":"2024-09-11T08:06:53.358070+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_clean_scrub_stamp":"2024-09-11T08:06:38.581279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:05:15.086841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"59'8","reported_seq":38,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:07:53.123133+0000","last_change":"2024-09-11T08:06:43.617812+0000","last_active":"2024-09-11T08:07:53.123133+0000","last_peered":"2024-09-11T08:07:53.123133+0000","last_clean":"2024-09-11T08:07:53.123133+0000","last_became_active":"2024-09-11T08:06:43.617529+0000","last_became_peered":"2024-09-11T08:06:43.617529+0000","last_unstale":"2024-09-11T08:07:53.123133+0000","last_undegraded":"2024-09-11T08:07:53.123133+0000","last_fullsized":"2024-09-11T08:07:53.123133+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_clean_scrub_stamp":"2024-09-11T08:06:42.602559+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:17:25.433041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.923190+0000","last_change":"2024-09-11T08:06:46.174033+0000","last_active":"2024-09-11T08:06:53.923190+0000","last_peered":"2024-09-11T08:06:53.923190+0000","last_clean":"2024-09-11T08:06:53.923190+0000","last_became_active":"2024-09-11T08:06:46.173667+0000","last_became_peered":"2024-09-11T08:06:46.173667+0000","last_unstale":"2024-09-11T08:06:53.923190+0000","last_undegraded":"2024-09-11T08:06:53.923190+0000","last_fullsized":"2024-09-11T08:06:53.923190+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_clean_scrub_stamp":"2024-09-11T08:06:44.606510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:06:17.689074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"59'12","reported_seq":39,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.355965+0000","last_change":"2024-09-11T08:06:41.620218+0000","last_active":"2024-09-11T08:06:53.355965+0000","last_peered":"2024-09-11T08:06:53.355965+0000","last_clean":"2024-09-11T08:06:53.355965+0000","last_became_active":"2024-09-11T08:06:41.619544+0000","last_became_peered":"2024-09-11T08:06:41.619544+0000","last_unstale":"2024-09-11T08:06:53.355965+0000","last_undegraded":"2024-09-11T08:06:53.355965+0000","last_fullsized":"2024-09-11T08:06:53.355965+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_clean_scrub_stamp":"2024-09-11T08:06:40.591817+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:49:09.046422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"52'3","reported_seq":44,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.356573+0000","last_change":"2024-09-11T08:06:39.607264+0000","last_active":"2024-09-11T08:06:53.356573+0000","last_peered":"2024-09-11T08:06:53.356573+0000","last_clean":"2024-09-11T08:06:53.356573+0000","last_became_active":"2024-09-11T08:06:39.606994+0000","last_became_peered":"2024-09-11T08:06:39.606994+0000","last_unstale":"2024-09-11T08:06:53.356573+0000","last_undegraded":"2024-09-11T08:06:53.356573+0000","last_fullsized":"2024-09-11T08:06:53.356573+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_clean_scrub_stamp":"2024-09-11T08:06:38.581279+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:29:01.420487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":23,"num_read_kb":23,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.356887+0000","last_change":"2024-09-11T08:06:43.622700+0000","last_active":"2024-09-11T08:06:53.356887+0000","last_peered":"2024-09-11T08:06:53.356887+0000","last_clean":"2024-09-11T08:06:53.356887+0000","last_became_active":"2024-09-11T08:06:43.622338+0000","last_became_peered":"2024-09-11T08:06:43.622338+0000","last_unstale":"2024-09-11T08:06:53.356887+0000","last_undegraded":"2024-09-11T08:06:53.356887+0000","last_fullsized":"2024-09-11T08:06:53.356887+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_clean_scrub_stamp":"2024-09-11T08:06:42.602559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:27:44.475248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.504909+0000","last_change":"2024-09-11T08:06:45.627104+0000","last_active":"2024-09-11T08:06:53.504909+0000","last_peered":"2024-09-11T08:06:53.504909+0000","last_clean":"2024-09-11T08:06:53.504909+0000","last_became_active":"2024-09-11T08:06:45.626776+0000","last_became_peered":"2024-09-11T08:06:45.626776+0000","last_unstale":"2024-09-11T08:06:53.504909+0000","last_undegraded":"2024-09-11T08:06:53.504909+0000","last_fullsized":"2024-09-11T08:06:53.504909+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_clean_scrub_stamp":"2024-09-11T08:06:44.606510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:27:09.664925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"59'9","reported_seq":32,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.358500+0000","last_change":"2024-09-11T08:06:42.464514+0000","last_active":"2024-09-11T08:06:53.358500+0000","last_peered":"2024-09-11T08:06:53.358500+0000","last_clean":"2024-09-11T08:06:53.358500+0000","last_became_active":"2024-09-11T08:06:42.463984+0000","last_became_peered":"2024-09-11T08:06:42.463984+0000","last_unstale":"2024-09-11T08:06:53.358500+0000","last_undegraded":"2024-09-11T08:06:53.358500+0000","last_fullsized":"2024-09-11T08:06:53.358500+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_clean_scrub_stamp":"2024-09-11T08:06:40.591817+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:22:31.967107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.356963+0000","last_change":"2024-09-11T08:06:39.604875+0000","last_active":"2024-09-11T08:06:53.356963+0000","last_peered":"2024-09-11T08:06:53.356963+0000","last_clean":"2024-09-11T08:06:53.356963+0000","last_became_active":"2024-09-11T08:06:39.604485+0000","last_became_peered":"2024-09-11T08:06:39.604485+0000","last_unstale":"2024-09-11T08:06:53.356963+0000","last_undegraded":"2024-09-11T08:06:53.356963+0000","last_fullsized":"2024-09-11T08:06:53.356963+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_clean_scrub_stamp":"2024-09-11T08:06:38.581279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:32:49.491454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.505524+0000","last_change":"2024-09-11T08:06:43.621732+0000","last_active":"2024-09-11T08:06:53.505524+0000","last_peered":"2024-09-11T08:06:53.505524+0000","last_clean":"2024-09-11T08:06:53.505524+0000","last_became_active":"2024-09-11T08:06:43.621463+0000","last_became_peered":"2024-09-11T08:06:43.621463+0000","last_unstale":"2024-09-11T08:06:53.505524+0000","last_undegraded":"2024-09-11T08:06:53.505524+0000","last_fullsized":"2024-09-11T08:06:53.505524+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_clean_scrub_stamp":"2024-09-11T08:06:42.602559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:03:39.556106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.358612+0000","last_change":"2024-09-11T08:06:45.628869+0000","last_active":"2024-09-11T08:06:53.358612+0000","last_peered":"2024-09-11T08:06:53.358612+0000","last_clean":"2024-09-11T08:06:53.358612+0000","last_became_active":"2024-09-11T08:06:45.628457+0000","last_became_peered":"2024-09-11T08:06:45.628457+0000","last_unstale":"2024-09-11T08:06:53.358612+0000","last_undegraded":"2024-09-11T08:06:53.358612+0000","last_fullsized":"2024-09-11T08:06:53.358612+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_clean_scrub_stamp":"2024-09-11T08:06:44.606510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:30:45.531791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.922153+0000","last_change":"2024-09-11T08:06:39.606516+0000","last_active":"2024-09-11T08:06:53.922153+0000","last_peered":"2024-09-11T08:06:53.922153+0000","last_clean":"2024-09-11T08:06:53.922153+0000","last_became_active":"2024-09-11T08:06:39.606173+0000","last_became_peered":"2024-09-11T08:06:39.606173+0000","last_unstale":"2024-09-11T08:06:53.922153+0000","last_undegraded":"2024-09-11T08:06:53.922153+0000","last_fullsized":"2024-09-11T08:06:53.922153+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_clean_scrub_stamp":"2024-09-11T08:06:38.581279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:42:30.212709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"59'6","reported_seq":25,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.505593+0000","last_change":"2024-09-11T08:06:42.463567+0000","last_active":"2024-09-11T08:06:53.505593+0000","last_peered":"2024-09-11T08:06:53.505593+0000","last_clean":"2024-09-11T08:06:53.505593+0000","last_became_active":"2024-09-11T08:06:42.463290+0000","last_became_peered":"2024-09-11T08:06:42.463290+0000","last_unstale":"2024-09-11T08:06:53.505593+0000","last_undegraded":"2024-09-11T08:06:53.505593+0000","last_fullsized":"2024-09-11T08:06:53.505593+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_clean_scrub_stamp":"2024-09-11T08:06:40.591817+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:34:30.894253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.923719+0000","last_change":"2024-09-11T08:06:43.627168+0000","last_active":"2024-09-11T08:06:53.923719+0000","last_peered":"2024-09-11T08:06:53.923719+0000","last_clean":"2024-09-11T08:06:53.923719+0000","last_became_active":"2024-09-11T08:06:43.626710+0000","last_became_peered":"2024-09-11T08:06:43.626710+0000","last_unstale":"2024-09-11T08:06:53.923719+0000","last_undegraded":"2024-09-11T08:06:53.923719+0000","last_fullsized":"2024-09-11T08:06:53.923719+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_clean_scrub_stamp":"2024-09-11T08:06:42.602559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:41:53.559126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.922464+0000","last_change":"2024-09-11T08:06:46.175979+0000","last_active":"2024-09-11T08:06:53.922464+0000","last_peered":"2024-09-11T08:06:53.922464+0000","last_clean":"2024-09-11T08:06:53.922464+0000","last_became_active":"2024-09-11T08:06:46.175692+0000","last_became_peered":"2024-09-11T08:06:46.175692+0000","last_unstale":"2024-09-11T08:06:53.922464+0000","last_undegraded":"2024-09-11T08:06:53.922464+0000","last_fullsized":"2024-09-11T08:06:53.922464+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_clean_scrub_stamp":"2024-09-11T08:06:44.606510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:54:55.090852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"59'10","reported_seq":31,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.505563+0000","last_change":"2024-09-11T08:06:42.463299+0000","last_active":"2024-09-11T08:06:53.505563+0000","last_peered":"2024-09-11T08:06:53.505563+0000","last_clean":"2024-09-11T08:06:53.505563+0000","last_became_active":"2024-09-11T08:06:42.462909+0000","last_became_peered":"2024-09-11T08:06:42.462909+0000","last_unstale":"2024-09-11T08:06:53.505563+0000","last_undegraded":"2024-09-11T08:06:53.505563+0000","last_fullsized":"2024-09-11T08:06:53.505563+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_clean_scrub_stamp":"2024-09-11T08:06:40.591817+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:52:15.756396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.923475+0000","last_change":"2024-09-11T08:06:39.605030+0000","last_active":"2024-09-11T08:06:53.923475+0000","last_peered":"2024-09-11T08:06:53.923475+0000","last_clean":"2024-09-11T08:06:53.923475+0000","last_became_active":"2024-09-11T08:06:39.604758+0000","last_became_peered":"2024-09-11T08:06:39.604758+0000","last_unstale":"2024-09-11T08:06:53.923475+0000","last_undegraded":"2024-09-11T08:06:53.923475+0000","last_fullsized":"2024-09-11T08:06:53.923475+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_clean_scrub_stamp":"2024-09-11T08:06:38.581279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:21:23.445399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"59'8","reported_seq":38,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:07:53.123135+0000","last_change":"2024-09-11T08:06:43.626946+0000","last_active":"2024-09-11T08:07:53.123135+0000","last_peered":"2024-09-11T08:07:53.123135+0000","last_clean":"2024-09-11T08:07:53.123135+0000","last_became_active":"2024-09-11T08:06:43.626013+0000","last_became_peered":"2024-09-11T08:06:43.626013+0000","last_unstale":"2024-09-11T08:07:53.123135+0000","last_undegraded":"2024-09-11T08:07:53.123135+0000","last_fullsized":"2024-09-11T08:07:53.123135+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_clean_scrub_stamp":"2024-09-11T08:06:42.602559+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:24:37.994554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.358182+0000","last_change":"2024-09-11T08:06:45.628724+0000","last_active":"2024-09-11T08:06:53.358182+0000","last_peered":"2024-09-11T08:06:53.358182+0000","last_clean":"2024-09-11T08:06:53.358182+0000","last_became_active":"2024-09-11T08:06:45.627922+0000","last_became_peered":"2024-09-11T08:06:45.627922+0000","last_unstale":"2024-09-11T08:06:53.358182+0000","last_undegraded":"2024-09-11T08:06:53.358182+0000","last_fullsized":"2024-09-11T08:06:53.358182+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_clean_scrub_stamp":"2024-09-11T08:06:44.606510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:51:02.980182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"59'9","reported_seq":32,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.923973+0000","last_change":"2024-09-11T08:06:42.464248+0000","last_active":"2024-09-11T08:06:53.923973+0000","last_peered":"2024-09-11T08:06:53.923973+0000","last_clean":"2024-09-11T08:06:53.923973+0000","last_became_active":"2024-09-11T08:06:42.463958+0000","last_became_peered":"2024-09-11T08:06:42.463958+0000","last_unstale":"2024-09-11T08:06:53.923973+0000","last_undegraded":"2024-09-11T08:06:53.923973+0000","last_fullsized":"2024-09-11T08:06:53.923973+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_clean_scrub_stamp":"2024-09-11T08:06:40.591817+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:38:51.336334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.358038+0000","last_change":"2024-09-11T08:06:39.607967+0000","last_active":"2024-09-11T08:06:53.358038+0000","last_peered":"2024-09-11T08:06:53.358038+0000","last_clean":"2024-09-11T08:06:53.358038+0000","last_became_active":"2024-09-11T08:06:39.607693+0000","last_became_peered":"2024-09-11T08:06:39.607693+0000","last_unstale":"2024-09-11T08:06:53.358038+0000","last_undegraded":"2024-09-11T08:06:53.358038+0000","last_fullsized":"2024-09-11T08:06:53.358038+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_clean_scrub_stamp":"2024-09-11T08:06:38.581279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:11.296892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"59'8","reported_seq":38,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:07:53.122896+0000","last_change":"2024-09-11T08:06:43.620860+0000","last_active":"2024-09-11T08:07:53.122896+0000","last_peered":"2024-09-11T08:07:53.122896+0000","last_clean":"2024-09-11T08:07:53.122896+0000","last_became_active":"2024-09-11T08:06:43.620572+0000","last_became_peered":"2024-09-11T08:06:43.620572+0000","last_unstale":"2024-09-11T08:07:53.122896+0000","last_undegraded":"2024-09-11T08:07:53.122896+0000","last_fullsized":"2024-09-11T08:07:53.122896+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_clean_scrub_stamp":"2024-09-11T08:06:42.602559+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:03:09.496213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.356486+0000","last_change":"2024-09-11T08:06:45.625629+0000","last_active":"2024-09-11T08:06:53.356486+0000","last_peered":"2024-09-11T08:06:53.356486+0000","last_clean":"2024-09-11T08:06:53.356486+0000","last_became_active":"2024-09-11T08:06:45.625334+0000","last_became_peered":"2024-09-11T08:06:45.625334+0000","last_unstale":"2024-09-11T08:06:53.356486+0000","last_undegraded":"2024-09-11T08:06:53.356486+0000","last_fullsized":"2024-09-11T08:06:53.356486+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_clean_scrub_stamp":"2024-09-11T08:06:44.606510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:52:42.198104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"59'9","reported_seq":32,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.922990+0000","last_change":"2024-09-11T08:06:41.618370+0000","last_active":"2024-09-11T08:06:53.922990+0000","last_peered":"2024-09-11T08:06:53.922990+0000","last_clean":"2024-09-11T08:06:53.922990+0000","last_became_active":"2024-09-11T08:06:41.616919+0000","last_became_peered":"2024-09-11T08:06:41.616919+0000","last_unstale":"2024-09-11T08:06:53.922990+0000","last_undegraded":"2024-09-11T08:06:53.922990+0000","last_fullsized":"2024-09-11T08:06:53.922990+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_clean_scrub_stamp":"2024-09-11T08:06:40.591817+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:44:01.768714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.922658+0000","last_change":"2024-09-11T08:06:39.601409+0000","last_active":"2024-09-11T08:06:53.922658+0000","last_peered":"2024-09-11T08:06:53.922658+0000","last_clean":"2024-09-11T08:06:53.922658+0000","last_became_active":"2024-09-11T08:06:39.600679+0000","last_became_peered":"2024-09-11T08:06:39.600679+0000","last_unstale":"2024-09-11T08:06:53.922658+0000","last_undegraded":"2024-09-11T08:06:53.922658+0000","last_fullsized":"2024-09-11T08:06:53.922658+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_clean_scrub_stamp":"2024-09-11T08:06:38.581279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:27:17.806820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.504985+0000","last_change":"2024-09-11T08:06:43.624960+0000","last_active":"2024-09-11T08:06:53.504985+0000","last_peered":"2024-09-11T08:06:53.504985+0000","last_clean":"2024-09-11T08:06:53.504985+0000","last_became_active":"2024-09-11T08:06:43.624654+0000","last_became_peered":"2024-09-11T08:06:43.624654+0000","last_unstale":"2024-09-11T08:06:53.504985+0000","last_undegraded":"2024-09-11T08:06:53.504985+0000","last_fullsized":"2024-09-11T08:06:53.504985+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_clean_scrub_stamp":"2024-09-11T08:06:42.602559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:35:56.009761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.356843+0000","last_change":"2024-09-11T08:06:45.630924+0000","last_active":"2024-09-11T08:06:53.356843+0000","last_peered":"2024-09-11T08:06:53.356843+0000","last_clean":"2024-09-11T08:06:53.356843+0000","last_became_active":"2024-09-11T08:06:45.630564+0000","last_became_peered":"2024-09-11T08:06:45.630564+0000","last_unstale":"2024-09-11T08:06:53.356843+0000","last_undegraded":"2024-09-11T08:06:53.356843+0000","last_fullsized":"2024-09-11T08:06:53.356843+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_clean_scrub_stamp":"2024-09-11T08:06:44.606510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:22:33.148769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"59'11","reported_seq":35,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.355982+0000","last_change":"2024-09-11T08:06:41.620509+0000","last_active":"2024-09-11T08:06:53.355982+0000","last_peered":"2024-09-11T08:06:53.355982+0000","last_clean":"2024-09-11T08:06:53.355982+0000","last_became_active":"2024-09-11T08:06:41.620024+0000","last_became_peered":"2024-09-11T08:06:41.620024+0000","last_unstale":"2024-09-11T08:06:53.355982+0000","last_undegraded":"2024-09-11T08:06:53.355982+0000","last_fullsized":"2024-09-11T08:06:53.355982+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_clean_scrub_stamp":"2024-09-11T08:06:40.591817+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:20.707096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.923522+0000","last_change":"2024-09-11T08:06:39.606984+0000","last_active":"2024-09-11T08:06:53.923522+0000","last_peered":"2024-09-11T08:06:53.923522+0000","last_clean":"2024-09-11T08:06:53.923522+0000","last_became_active":"2024-09-11T08:06:39.606710+0000","last_became_peered":"2024-09-11T08:06:39.606710+0000","last_unstale":"2024-09-11T08:06:53.923522+0000","last_undegraded":"2024-09-11T08:06:53.923522+0000","last_fullsized":"2024-09-11T08:06:53.923522+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_clean_scrub_stamp":"2024-09-11T08:06:38.581279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:45:54.406289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.923060+0000","last_change":"2024-09-11T08:06:43.629521+0000","last_active":"2024-09-11T08:06:53.923060+0000","last_peered":"2024-09-11T08:06:53.923060+0000","last_clean":"2024-09-11T08:06:53.923060+0000","last_became_active":"2024-09-11T08:06:43.629214+0000","last_became_peered":"2024-09-11T08:06:43.629214+0000","last_unstale":"2024-09-11T08:06:53.923060+0000","last_undegraded":"2024-09-11T08:06:53.923060+0000","last_fullsized":"2024-09-11T08:06:53.923060+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_clean_scrub_stamp":"2024-09-11T08:06:42.602559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:38:34.846754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.504636+0000","last_change":"2024-09-11T08:06:45.630229+0000","last_active":"2024-09-11T08:06:53.504636+0000","last_peered":"2024-09-11T08:06:53.504636+0000","last_clean":"2024-09-11T08:06:53.504636+0000","last_became_active":"2024-09-11T08:06:45.629929+0000","last_became_peered":"2024-09-11T08:06:45.629929+0000","last_unstale":"2024-09-11T08:06:53.504636+0000","last_undegraded":"2024-09-11T08:06:53.504636+0000","last_fullsized":"2024-09-11T08:06:53.504636+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_clean_scrub_stamp":"2024-09-11T08:06:44.606510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:18:12.022222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"59'4","reported_seq":22,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.504677+0000","last_change":"2024-09-11T08:06:41.621102+0000","last_active":"2024-09-11T08:06:53.504677+0000","last_peered":"2024-09-11T08:06:53.504677+0000","last_clean":"2024-09-11T08:06:53.504677+0000","last_became_active":"2024-09-11T08:06:41.620784+0000","last_became_peered":"2024-09-11T08:06:41.620784+0000","last_unstale":"2024-09-11T08:06:53.504677+0000","last_undegraded":"2024-09-11T08:06:53.504677+0000","last_fullsized":"2024-09-11T08:06:53.504677+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_clean_scrub_stamp":"2024-09-11T08:06:40.591817+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:59:40.715245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.923844+0000","last_change":"2024-09-11T08:06:39.605989+0000","last_active":"2024-09-11T08:06:53.923844+0000","last_peered":"2024-09-11T08:06:53.923844+0000","last_clean":"2024-09-11T08:06:53.923844+0000","last_became_active":"2024-09-11T08:06:39.605610+0000","last_became_peered":"2024-09-11T08:06:39.605610+0000","last_unstale":"2024-09-11T08:06:53.923844+0000","last_undegraded":"2024-09-11T08:06:53.923844+0000","last_fullsized":"2024-09-11T08:06:53.923844+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_clean_scrub_stamp":"2024-09-11T08:06:38.581279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:29:10.795022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.921898+0000","last_change":"2024-09-11T08:06:43.621216+0000","last_active":"2024-09-11T08:06:53.921898+0000","last_peered":"2024-09-11T08:06:53.921898+0000","last_clean":"2024-09-11T08:06:53.921898+0000","last_became_active":"2024-09-11T08:06:43.620960+0000","last_became_peered":"2024-09-11T08:06:43.620960+0000","last_unstale":"2024-09-11T08:06:53.921898+0000","last_undegraded":"2024-09-11T08:06:53.921898+0000","last_fullsized":"2024-09-11T08:06:53.921898+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_clean_scrub_stamp":"2024-09-11T08:06:42.602559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:10:15.343370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.921911+0000","last_change":"2024-09-11T08:06:45.628406+0000","last_active":"2024-09-11T08:06:53.921911+0000","last_peered":"2024-09-11T08:06:53.921911+0000","last_clean":"2024-09-11T08:06:53.921911+0000","last_became_active":"2024-09-11T08:06:45.627358+0000","last_became_peered":"2024-09-11T08:06:45.627358+0000","last_unstale":"2024-09-11T08:06:53.921911+0000","last_undegraded":"2024-09-11T08:06:53.921911+0000","last_fullsized":"2024-09-11T08:06:53.921911+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_clean_scrub_stamp":"2024-09-11T08:06:44.606510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:12:15.432613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"59'11","reported_seq":35,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.504912+0000","last_change":"2024-09-11T08:06:42.464194+0000","last_active":"2024-09-11T08:06:53.504912+0000","last_peered":"2024-09-11T08:06:53.504912+0000","last_clean":"2024-09-11T08:06:53.504912+0000","last_became_active":"2024-09-11T08:06:42.463040+0000","last_became_peered":"2024-09-11T08:06:42.463040+0000","last_unstale":"2024-09-11T08:06:53.504912+0000","last_undegraded":"2024-09-11T08:06:53.504912+0000","last_fullsized":"2024-09-11T08:06:53.504912+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_clean_scrub_stamp":"2024-09-11T08:06:40.591817+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:20:31.230643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.922924+0000","last_change":"2024-09-11T08:06:39.603395+0000","last_active":"2024-09-11T08:06:53.922924+0000","last_peered":"2024-09-11T08:06:53.922924+0000","last_clean":"2024-09-11T08:06:53.922924+0000","last_became_active":"2024-09-11T08:06:39.602940+0000","last_became_peered":"2024-09-11T08:06:39.602940+0000","last_unstale":"2024-09-11T08:06:53.922924+0000","last_undegraded":"2024-09-11T08:06:53.922924+0000","last_fullsized":"2024-09-11T08:06:53.922924+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_clean_scrub_stamp":"2024-09-11T08:06:38.581279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:20:13.814946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.922672+0000","last_change":"2024-09-11T08:06:43.619135+0000","last_active":"2024-09-11T08:06:53.922672+0000","last_peered":"2024-09-11T08:06:53.922672+0000","last_clean":"2024-09-11T08:06:53.922672+0000","last_became_active":"2024-09-11T08:06:43.618865+0000","last_became_peered":"2024-09-11T08:06:43.618865+0000","last_unstale":"2024-09-11T08:06:53.922672+0000","last_undegraded":"2024-09-11T08:06:53.922672+0000","last_fullsized":"2024-09-11T08:06:53.922672+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_clean_scrub_stamp":"2024-09-11T08:06:42.602559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:51:03.011293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.504743+0000","last_change":"2024-09-11T08:06:46.174680+0000","last_active":"2024-09-11T08:06:53.504743+0000","last_peered":"2024-09-11T08:06:53.504743+0000","last_clean":"2024-09-11T08:06:53.504743+0000","last_became_active":"2024-09-11T08:06:46.174359+0000","last_became_peered":"2024-09-11T08:06:46.174359+0000","last_unstale":"2024-09-11T08:06:53.504743+0000","last_undegraded":"2024-09-11T08:06:53.504743+0000","last_fullsized":"2024-09-11T08:06:53.504743+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_clean_scrub_stamp":"2024-09-11T08:06:44.606510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:19:21.822289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"59'1","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.921790+0000","last_change":"2024-09-11T08:06:45.628156+0000","last_active":"2024-09-11T08:06:53.921790+0000","last_peered":"2024-09-11T08:06:53.921790+0000","last_clean":"2024-09-11T08:06:53.921790+0000","last_became_active":"2024-09-11T08:06:45.627877+0000","last_became_peered":"2024-09-11T08:06:45.627877+0000","last_unstale":"2024-09-11T08:06:53.921790+0000","last_undegraded":"2024-09-11T08:06:53.921790+0000","last_fullsized":"2024-09-11T08:06:53.921790+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_clean_scrub_stamp":"2024-09-11T08:06:44.606510+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:28:49.200951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.923962+0000","last_change":"2024-09-11T08:06:39.604005+0000","last_active":"2024-09-11T08:06:53.923962+0000","last_peered":"2024-09-11T08:06:53.923962+0000","last_clean":"2024-09-11T08:06:53.923962+0000","last_became_active":"2024-09-11T08:06:39.603733+0000","last_became_peered":"2024-09-11T08:06:39.603733+0000","last_unstale":"2024-09-11T08:06:53.923962+0000","last_undegraded":"2024-09-11T08:06:53.923962+0000","last_fullsized":"2024-09-11T08:06:53.923962+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_clean_scrub_stamp":"2024-09-11T08:06:38.581279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:01.485991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"59'10","reported_seq":31,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.358436+0000","last_change":"2024-09-11T08:06:42.464747+0000","last_active":"2024-09-11T08:06:53.358436+0000","last_peered":"2024-09-11T08:06:53.358436+0000","last_clean":"2024-09-11T08:06:53.358436+0000","last_became_active":"2024-09-11T08:06:42.464432+0000","last_became_peered":"2024-09-11T08:06:42.464432+0000","last_unstale":"2024-09-11T08:06:53.358436+0000","last_undegraded":"2024-09-11T08:06:53.358436+0000","last_fullsized":"2024-09-11T08:06:53.358436+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_clean_scrub_stamp":"2024-09-11T08:06:40.591817+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:46:26.124072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"59'8","reported_seq":41,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:07:53.123124+0000","last_change":"2024-09-11T08:06:43.619638+0000","last_active":"2024-09-11T08:07:53.123124+0000","last_peered":"2024-09-11T08:07:53.123124+0000","last_clean":"2024-09-11T08:07:53.123124+0000","last_became_active":"2024-09-11T08:06:43.619337+0000","last_became_peered":"2024-09-11T08:06:43.619337+0000","last_unstale":"2024-09-11T08:07:53.123124+0000","last_undegraded":"2024-09-11T08:07:53.123124+0000","last_fullsized":"2024-09-11T08:07:53.123124+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_clean_scrub_stamp":"2024-09-11T08:06:42.602559+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:55:38.708691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.922291+0000","last_change":"2024-09-11T08:06:45.627949+0000","last_active":"2024-09-11T08:06:53.922291+0000","last_peered":"2024-09-11T08:06:53.922291+0000","last_clean":"2024-09-11T08:06:53.922291+0000","last_became_active":"2024-09-11T08:06:45.627633+0000","last_became_peered":"2024-09-11T08:06:45.627633+0000","last_unstale":"2024-09-11T08:06:53.922291+0000","last_undegraded":"2024-09-11T08:06:53.922291+0000","last_fullsized":"2024-09-11T08:06:53.922291+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_clean_scrub_stamp":"2024-09-11T08:06:44.606510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:34:40.595042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.505251+0000","last_change":"2024-09-11T08:06:39.605550+0000","last_active":"2024-09-11T08:06:53.505251+0000","last_peered":"2024-09-11T08:06:53.505251+0000","last_clean":"2024-09-11T08:06:53.505251+0000","last_became_active":"2024-09-11T08:06:39.605246+0000","last_became_peered":"2024-09-11T08:06:39.605246+0000","last_unstale":"2024-09-11T08:06:53.505251+0000","last_undegraded":"2024-09-11T08:06:53.505251+0000","last_fullsized":"2024-09-11T08:06:53.505251+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_clean_scrub_stamp":"2024-09-11T08:06:38.581279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:02:04.921114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"59'11","reported_seq":35,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.922854+0000","last_change":"2024-09-11T08:06:41.620114+0000","last_active":"2024-09-11T08:06:53.922854+0000","last_peered":"2024-09-11T08:06:53.922854+0000","last_clean":"2024-09-11T08:06:53.922854+0000","last_became_active":"2024-09-11T08:06:41.619861+0000","last_became_peered":"2024-09-11T08:06:41.619861+0000","last_unstale":"2024-09-11T08:06:53.922854+0000","last_undegraded":"2024-09-11T08:06:53.922854+0000","last_fullsized":"2024-09-11T08:06:53.922854+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_clean_scrub_stamp":"2024-09-11T08:06:40.591817+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:50:47.601811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.358352+0000","last_change":"2024-09-11T08:06:43.614213+0000","last_active":"2024-09-11T08:06:53.358352+0000","last_peered":"2024-09-11T08:06:53.358352+0000","last_clean":"2024-09-11T08:06:53.358352+0000","last_became_active":"2024-09-11T08:06:43.613927+0000","last_became_peered":"2024-09-11T08:06:43.613927+0000","last_unstale":"2024-09-11T08:06:53.358352+0000","last_undegraded":"2024-09-11T08:06:53.358352+0000","last_fullsized":"2024-09-11T08:06:53.358352+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_clean_scrub_stamp":"2024-09-11T08:06:42.602559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:15:07.385448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.356268+0000","last_change":"2024-09-11T08:06:46.172587+0000","last_active":"2024-09-11T08:06:53.356268+0000","last_peered":"2024-09-11T08:06:53.356268+0000","last_clean":"2024-09-11T08:06:53.356268+0000","last_became_active":"2024-09-11T08:06:46.172235+0000","last_became_peered":"2024-09-11T08:06:46.172235+0000","last_unstale":"2024-09-11T08:06:53.356268+0000","last_undegraded":"2024-09-11T08:06:53.356268+0000","last_fullsized":"2024-09-11T08:06:53.356268+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_clean_scrub_stamp":"2024-09-11T08:06:44.606510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:13:30.787527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.922892+0000","last_change":"2024-09-11T08:06:39.600828+0000","last_active":"2024-09-11T08:06:53.922892+0000","last_peered":"2024-09-11T08:06:53.922892+0000","last_clean":"2024-09-11T08:06:53.922892+0000","last_became_active":"2024-09-11T08:06:39.600572+0000","last_became_peered":"2024-09-11T08:06:39.600572+0000","last_unstale":"2024-09-11T08:06:53.922892+0000","last_undegraded":"2024-09-11T08:06:53.922892+0000","last_fullsized":"2024-09-11T08:06:53.922892+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_clean_scrub_stamp":"2024-09-11T08:06:38.581279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:39:20.392722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"59'15","reported_seq":41,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.358968+0000","last_change":"2024-09-11T08:06:41.618530+0000","last_active":"2024-09-11T08:06:53.358968+0000","last_peered":"2024-09-11T08:06:53.358968+0000","last_clean":"2024-09-11T08:06:53.358968+0000","last_became_active":"2024-09-11T08:06:41.618210+0000","last_became_peered":"2024-09-11T08:06:41.618210+0000","last_unstale":"2024-09-11T08:06:53.358968+0000","last_undegraded":"2024-09-11T08:06:53.358968+0000","last_fullsized":"2024-09-11T08:06:53.358968+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_clean_scrub_stamp":"2024-09-11T08:06:40.591817+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:21:39.506662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.923351+0000","last_change":"2024-09-11T08:06:43.623753+0000","last_active":"2024-09-11T08:06:53.923351+0000","last_peered":"2024-09-11T08:06:53.923351+0000","last_clean":"2024-09-11T08:06:53.923351+0000","last_became_active":"2024-09-11T08:06:43.623433+0000","last_became_peered":"2024-09-11T08:06:43.623433+0000","last_unstale":"2024-09-11T08:06:53.923351+0000","last_undegraded":"2024-09-11T08:06:53.923351+0000","last_fullsized":"2024-09-11T08:06:53.923351+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_clean_scrub_stamp":"2024-09-11T08:06:42.602559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:39:51.475323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.505402+0000","last_change":"2024-09-11T08:06:46.172978+0000","last_active":"2024-09-11T08:06:53.505402+0000","last_peered":"2024-09-11T08:06:53.505402+0000","last_clean":"2024-09-11T08:06:53.505402+0000","last_became_active":"2024-09-11T08:06:46.172348+0000","last_became_peered":"2024-09-11T08:06:46.172348+0000","last_unstale":"2024-09-11T08:06:53.505402+0000","last_undegraded":"2024-09-11T08:06:53.505402+0000","last_fullsized":"2024-09-11T08:06:53.505402+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_clean_scrub_stamp":"2024-09-11T08:06:44.606510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:03:05.325184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.505132+0000","last_change":"2024-09-11T08:06:39.602456+0000","last_active":"2024-09-11T08:06:53.505132+0000","last_peered":"2024-09-11T08:06:53.505132+0000","last_clean":"2024-09-11T08:06:53.505132+0000","last_became_active":"2024-09-11T08:06:39.601844+0000","last_became_peered":"2024-09-11T08:06:39.601844+0000","last_unstale":"2024-09-11T08:06:53.505132+0000","last_undegraded":"2024-09-11T08:06:53.505132+0000","last_fullsized":"2024-09-11T08:06:53.505132+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_clean_scrub_stamp":"2024-09-11T08:06:38.581279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:45:49.618077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"59'12","reported_seq":39,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.505247+0000","last_change":"2024-09-11T08:06:41.620881+0000","last_active":"2024-09-11T08:06:53.505247+0000","last_peered":"2024-09-11T08:06:53.505247+0000","last_clean":"2024-09-11T08:06:53.505247+0000","last_became_active":"2024-09-11T08:06:41.620564+0000","last_became_peered":"2024-09-11T08:06:41.620564+0000","last_unstale":"2024-09-11T08:06:53.505247+0000","last_undegraded":"2024-09-11T08:06:53.505247+0000","last_fullsized":"2024-09-11T08:06:53.505247+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_clean_scrub_stamp":"2024-09-11T08:06:40.591817+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:52:01.769772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.356073+0000","last_change":"2024-09-11T08:06:43.623022+0000","last_active":"2024-09-11T08:06:53.356073+0000","last_peered":"2024-09-11T08:06:53.356073+0000","last_clean":"2024-09-11T08:06:53.356073+0000","last_became_active":"2024-09-11T08:06:43.622423+0000","last_became_peered":"2024-09-11T08:06:43.622423+0000","last_unstale":"2024-09-11T08:06:53.356073+0000","last_undegraded":"2024-09-11T08:06:53.356073+0000","last_fullsized":"2024-09-11T08:06:53.356073+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_clean_scrub_stamp":"2024-09-11T08:06:42.602559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:50:27.237673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":391,"ondisk_log_size":391,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":77,"num_read_kb":72,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":48,"seq":206158430230,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298600,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446552,"statfs":{"total":95995035648,"available":95689269248,"internally_reserved":0,"allocated":1277952,"data_stored":868928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:07:28 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"last":0.68100000000000005}]},{"osd":1,"last update":"Wed Sep 11 08:07:28 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.54700000000000004}]},{"osd":2,"last update":"Wed Sep 11 08:07:28 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.71199999999999997}]},{"osd":3,"last update":"Wed Sep 11 08:07:28 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.625},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.434}]},{"osd":4,"last update":"Wed Sep 11 08:07:28 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.73199999999999998}]},{"osd":5,"last update":"Wed Sep 11 08:07:28 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.57999999999999996}]},{"osd":6,"last update":"Wed Sep 11 08:07:28 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.60999999999999999}]}]},{"osd":6,"up_from":42,"seq":180388626458,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298576,"kb_used_data":1224,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446576,"statfs":{"total":95995035648,"available":95689293824,"internally_reserved":0,"allocated":1253376,"data_stored":867406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:07:03 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.40999999999999998}]},{"osd":1,"last update":"Wed Sep 11 08:07:03 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.312},{"interface":"front","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.55200000000000005}]},{"osd":2,"last update":"Wed Sep 11 08:07:03 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.45200000000000001}]},{"osd":3,"last update":"Wed Sep 11 08:07:03 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.36699999999999999}]},{"osd":4,"last update":"Wed Sep 11 08:07:03 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.56799999999999995}]},{"osd":5,"last update":"Wed Sep 11 08:07:03 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.432,"15min":0.432},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.433,"5min":0.433,"15min":0.433},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.53200000000000003}]},{"osd":7,"last update":"Wed Sep 11 08:07:28 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.48799999999999999}]}]},{"osd":5,"up_from":36,"seq":154618822688,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297996,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447156,"statfs":{"total":95995035648,"available":95689887744,"internally_reserved":0,"allocated":655360,"data_stored":276659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:07:44 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.51000000000000001,"15min":0.51600000000000001},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.53200000000000003,"15min":0.53600000000000003},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.69199999999999995,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.41499999999999998}]},{"osd":1,"last update":"Wed Sep 11 08:07:44 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.52400000000000002,"15min":0.53100000000000003},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.64800000000000002,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.34399999999999997}]},{"osd":2,"last update":"Wed Sep 11 08:07:44 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.55700000000000005,"15min":0.56499999999999995},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.68000000000000005,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.496,"5min":0.504,"15min":0.505},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.52700000000000002}]},{"osd":3,"last update":"Wed Sep 11 08:07:44 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.67400000000000004,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.434},{"interface":"front","average":{"1min":0.503,"5min":0.54200000000000004,"15min":0.54800000000000004},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.72099999999999997,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.59699999999999998}]},{"osd":4,"last update":"Wed Sep 11 08:07:44 2024","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.36799999999999999,"15min":0.35899999999999999},"min":{"1min":0.29199999999999998,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.39100000000000001,"15min":0.375},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.56000000000000005}]},{"osd":6,"last update":"Wed Sep 11 08:07:05 2024","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.628},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.53900000000000003}]},{"osd":7,"last update":"Wed Sep 11 08:07:27 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"last":0.48999999999999999}]}]},{"osd":4,"up_from":30,"seq":128849018916,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298020,"kb_used_data":664,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447132,"statfs":{"total":95995035648,"available":95689863168,"internally_reserved":0,"allocated":679936,"data_stored":278036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:07:16 2024","interfaces":[{"interface":"back","average":{"1min":1.3169999999999999,"5min":0.68100000000000005,"15min":0.57499999999999996},"min":{"1min":0.35699999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":19.065999999999999,"5min":19.065999999999999,"15min":19.065999999999999},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.51500000000000001,"15min":0.51400000000000001},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.52300000000000002}]},{"osd":1,"last update":"Wed Sep 11 08:07:16 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.503,"15min":0.504},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.68100000000000005,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.53300000000000003,"15min":0.53700000000000003},"min":{"1min":0.39300000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.74199999999999999,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.45200000000000001}]},{"osd":2,"last update":"Wed Sep 11 08:07:16 2024","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":0.67200000000000004,"15min":0.56499999999999995},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":19.041,"5min":19.041,"15min":19.041},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.51500000000000001,"15min":0.51300000000000001},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.63600000000000001}]},{"osd":3,"last update":"Wed Sep 11 08:07:16 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.52900000000000003,"15min":0.52500000000000002},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.505,"15min":0.503},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.71599999999999997,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.54300000000000004}]},{"osd":5,"last update":"Wed Sep 11 08:07:40 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.36299999999999999,"15min":0.35199999999999998},"min":{"1min":0.28000000000000003,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.36599999999999999,"15min":0.35599999999999998},"min":{"1min":0.313,"5min":0.253,"15min":0.253},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.38200000000000001}]},{"osd":6,"last update":"Wed Sep 11 08:07:05 2024","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.46700000000000003}]},{"osd":7,"last update":"Wed Sep 11 08:07:30 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.194,"5min":1.194,"15min":1.194},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.65700000000000003}]}]},{"osd":3,"up_from":25,"seq":107374182441,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298036,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447116,"statfs":{"total":95995035648,"available":95689846784,"internally_reserved":0,"allocated":696320,"data_stored":277526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:07:59 2024","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.37,"15min":0.374},"min":{"1min":0.193,"5min":0.193,"15min":0.193},"max":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.37,"5min":0.39700000000000002,"15min":0.40500000000000003},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.49099999999999999}]},{"osd":1,"last update":"Wed Sep 11 08:07:59 2024","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.40799999999999997,"15min":0.41099999999999998},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.41299999999999998,"15min":0.42699999999999999},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.60399999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.28699999999999998}]},{"osd":2,"last update":"Wed Sep 11 08:07:59 2024","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.44800000000000001,"15min":0.46000000000000002},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.43099999999999999,"15min":0.439},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.307}]},{"osd":4,"last update":"Wed Sep 11 08:07:21 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.379},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.56699999999999995,"15min":0.56100000000000005},"min":{"1min":0.48299999999999998,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.52400000000000002}]},{"osd":5,"last update":"Wed Sep 11 08:07:44 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.58499999999999996,"15min":0.58899999999999997},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.621,"5min":0.60699999999999998,"15min":0.60499999999999998},"min":{"1min":0.42999999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.51400000000000001}]},{"osd":6,"last update":"Wed Sep 11 08:07:05 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.441},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.50700000000000001}]},{"osd":7,"last update":"Wed Sep 11 08:07:28 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.46800000000000003}]}]},{"osd":2,"up_from":18,"seq":77309411373,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298000,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447152,"statfs":{"total":95995035648,"available":95689883648,"internally_reserved":0,"allocated":659456,"data_stored":277165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:07:34 2024","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.35999999999999999,"15min":0.35599999999999998},"min":{"1min":0.255,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"last":0.22700000000000001},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.372,"15min":0.36899999999999999},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.29399999999999998}]},{"osd":1,"last update":"Wed Sep 11 08:07:34 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.39400000000000002,"15min":0.38600000000000001},"min":{"1min":0.27200000000000002,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.311},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.39300000000000002,"15min":0.39300000000000002},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.23999999999999999}]},{"osd":3,"last update":"Wed Sep 11 08:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.40799999999999997,"15min":0.41199999999999998},"min":{"1min":0.26400000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.47999999999999998,"5min":0.626,"15min":0.626},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.44400000000000001,"15min":0.45000000000000001},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.60199999999999998,"5min":0.624,"15min":0.624},"last":0.35599999999999998}]},{"osd":4,"last update":"Wed Sep 11 08:07:20 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.52400000000000002,"15min":0.52700000000000002},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.71499999999999997,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.56499999999999995,"15min":0.56899999999999995},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.68899999999999995,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.43099999999999999}]},{"osd":5,"last update":"Wed Sep 11 08:07:42 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.53300000000000003,"15min":0.53100000000000003},"min":{"1min":0.44800000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.66200000000000003,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.55000000000000004,"15min":0.54800000000000004},"min":{"1min":0.42499999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.68999999999999995,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.51400000000000001}]},{"osd":6,"last update":"Wed Sep 11 08:07:06 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.373},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.42299999999999999}]},{"osd":7,"last update":"Wed Sep 11 08:07:28 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.44}]}]},{"osd":1,"up_from":13,"seq":55834574899,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298052,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447100,"statfs":{"total":95995035648,"available":95689830400,"internally_reserved":0,"allocated":712704,"data_stored":278685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:07:14 2024","interfaces":[{"interface":"back","average":{"1min":0.32300000000000001,"5min":0.32700000000000001,"15min":0.316},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.53600000000000003,"5min":0.60899999999999999,"15min":0.60899999999999999},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.34399999999999997,"15min":0.32400000000000001},"min":{"1min":0.24199999999999999,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.52200000000000002,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.27500000000000002}]},{"osd":2,"last update":"Wed Sep 11 08:07:32 2024","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.501,"5min":0.52200000000000002,"15min":0.52200000000000002},"last":0.38},{"interface":"front","average":{"1min":0.38,"5min":0.39600000000000002,"15min":0.39800000000000002},"min":{"1min":0.30599999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.56000000000000005,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.32400000000000001}]},{"osd":3,"last update":"Wed Sep 11 08:07:00 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.44500000000000001,"15min":0.45100000000000001},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.54400000000000004,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.436,"15min":0.44400000000000001},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.53200000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.309}]},{"osd":4,"last update":"Wed Sep 11 08:07:20 2024","interfaces":[{"interface":"back","average":{"1min":1.419,"5min":0.73799999999999999,"15min":0.624},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":17.425000000000001,"5min":17.425000000000001,"15min":17.425000000000001},"last":0.443},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.56100000000000005,"15min":0.56200000000000006},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.93700000000000006,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.47099999999999997}]},{"osd":5,"last update":"Wed Sep 11 08:07:42 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.79700000000000004,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.60899999999999999,"15min":0.61099999999999999},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.78300000000000003,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.41099999999999998}]},{"osd":6,"last update":"Wed Sep 11 08:07:04 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.51700000000000002},{"interface":"front","average":{"1min":1.403,"5min":1.403,"15min":1.403},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":17.452000000000002,"5min":17.452000000000002,"15min":17.452000000000002},"last":0.42099999999999999}]},{"osd":7,"last update":"Wed Sep 11 08:07:32 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.43099999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738424,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298592,"kb_used_data":1240,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446560,"statfs":{"total":95995035648,"available":95689277440,"internally_reserved":0,"allocated":1269760,"data_stored":868297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 11 08:07:13 2024","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.33400000000000002,"15min":0.33400000000000002},"min":{"1min":0.17199999999999999,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":0.41999999999999998,"5min":0.53400000000000003,"15min":0.53400000000000003},"last":0.26300000000000001},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.35499999999999998,"15min":0.34799999999999998},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.48199999999999998,"5min":0.499,"15min":0.499},"last":0.20100000000000001}]},{"osd":2,"last update":"Wed Sep 11 08:07:30 2024","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.38200000000000001,"15min":0.38800000000000001},"min":{"1min":0.22700000000000001,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.376,"15min":0.38200000000000001},"min":{"1min":0.252,"5min":0.249,"15min":0.249},"max":{"1min":0.496,"5min":0.52700000000000002,"15min":0.52700000000000002},"last":0.245}]},{"osd":3,"last update":"Wed Sep 11 08:07:58 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.41299999999999998,"15min":0.42599999999999999},"min":{"1min":0.28199999999999997,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.54100000000000004,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.31},{"interface":"front","average":{"1min":0.376,"5min":0.39700000000000002,"15min":0.40699999999999997},"min":{"1min":0.222,"5min":0.221,"15min":0.221},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.223}]},{"osd":4,"last update":"Wed Sep 11 08:07:19 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.52000000000000002,"15min":0.52400000000000002},"min":{"1min":0.35499999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55800000000000005,"15min":0.56000000000000005},"min":{"1min":0.44400000000000001,"5min":0.436,"15min":0.436},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.38300000000000001}]},{"osd":5,"last update":"Wed Sep 11 08:07:44 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.55500000000000005,"15min":0.52200000000000002},"min":{"1min":0.41599999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":4.5490000000000004,"5min":4.5490000000000004,"15min":4.5490000000000004},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.40899999999999997,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.40600000000000003}]},{"osd":6,"last update":"Wed Sep 11 08:07:04 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.45500000000000002}]},{"osd":7,"last update":"Wed Sep 11 08:07:28 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.42499999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T08:08:05.284 DEBUG:teuthology.orchestra.run.smithi122:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid f1d75850-7013-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T08:08:05.529 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:08:06.560 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:06 smithi122 ceph-mon[39726]: from='client.14679 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:08:06.560 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:06 smithi122 ceph-mon[39726]: pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:08:06.560 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:06 smithi122 ceph-mon[28992]: from='client.14679 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:08:06.560 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:06 smithi122 ceph-mon[28992]: pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:08:06.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:06 smithi135 ceph-mon[32689]: from='client.14679 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:08:06.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:06 smithi135 ceph-mon[32689]: pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:08:07.273 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T08:08:07.275 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T08:08:07.907 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2024-09-11T08:08:07.069867+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596188,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":930,"num_read_kb":787,"num_write":543,"num_write_kb":1671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":548,"ondisk_log_size":548,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2385872,"kb_used_data":7036,"kb_used_omap":13,"kb_used_meta":2378738,"kb_avail":747575344,"statfs":{"total":767960285184,"available":765517152256,"internally_reserved":0,"allocated":7204864,"data_stored":3992702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2435828416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002840"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.504810+0000","last_change":"2024-09-11T08:06:46.174781+0000","last_active":"2024-09-11T08:06:53.504810+0000","last_peered":"2024-09-11T08:06:53.504810+0000","last_clean":"2024-09-11T08:06:53.504810+0000","last_became_active":"2024-09-11T08:06:46.174393+0000","last_became_peered":"2024-09-11T08:06:46.174393+0000","last_unstale":"2024-09-11T08:06:53.504810+0000","last_undegraded":"2024-09-11T08:06:53.504810+0000","last_fullsized":"2024-09-11T08:06:53.504810+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_clean_scrub_stamp":"2024-09-11T08:06:44.606510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:31:08.409161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.357679+0000","last_change":"2024-09-11T08:06:39.610840+0000","last_active":"2024-09-11T08:06:53.357679+0000","last_peered":"2024-09-11T08:06:53.357679+0000","last_clean":"2024-09-11T08:06:53.357679+0000","last_became_active":"2024-09-11T08:06:39.610598+0000","last_became_peered":"2024-09-11T08:06:39.610598+0000","last_unstale":"2024-09-11T08:06:53.357679+0000","last_undegraded":"2024-09-11T08:06:53.357679+0000","last_fullsized":"2024-09-11T08:06:53.357679+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_clean_scrub_stamp":"2024-09-11T08:06:38.581279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:24:22.740831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"59'15","reported_seq":41,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.505130+0000","last_change":"2024-09-11T08:06:41.612891+0000","last_active":"2024-09-11T08:06:53.505130+0000","last_peered":"2024-09-11T08:06:53.505130+0000","last_clean":"2024-09-11T08:06:53.505130+0000","last_became_active":"2024-09-11T08:06:41.612600+0000","last_became_peered":"2024-09-11T08:06:41.612600+0000","last_unstale":"2024-09-11T08:06:53.505130+0000","last_undegraded":"2024-09-11T08:06:53.505130+0000","last_fullsized":"2024-09-11T08:06:53.505130+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_clean_scrub_stamp":"2024-09-11T08:06:40.591817+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:03:34.211085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.355780+0000","last_change":"2024-09-11T08:06:43.625089+0000","last_active":"2024-09-11T08:06:53.355780+0000","last_peered":"2024-09-11T08:06:53.355780+0000","last_clean":"2024-09-11T08:06:53.355780+0000","last_became_active":"2024-09-11T08:06:43.624550+0000","last_became_peered":"2024-09-11T08:06:43.624550+0000","last_unstale":"2024-09-11T08:06:53.355780+0000","last_undegraded":"2024-09-11T08:06:53.355780+0000","last_fullsized":"2024-09-11T08:06:53.355780+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_clean_scrub_stamp":"2024-09-11T08:06:42.602559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:45:31.954738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.505573+0000","last_change":"2024-09-11T08:06:39.606782+0000","last_active":"2024-09-11T08:06:53.505573+0000","last_peered":"2024-09-11T08:06:53.505573+0000","last_clean":"2024-09-11T08:06:53.505573+0000","last_became_active":"2024-09-11T08:06:39.606496+0000","last_became_peered":"2024-09-11T08:06:39.606496+0000","last_unstale":"2024-09-11T08:06:53.505573+0000","last_undegraded":"2024-09-11T08:06:53.505573+0000","last_fullsized":"2024-09-11T08:06:53.505573+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_clean_scrub_stamp":"2024-09-11T08:06:38.581279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:14:50.505270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"59'9","reported_seq":32,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.355788+0000","last_change":"2024-09-11T08:06:41.615154+0000","last_active":"2024-09-11T08:06:53.355788+0000","last_peered":"2024-09-11T08:06:53.355788+0000","last_clean":"2024-09-11T08:06:53.355788+0000","last_became_active":"2024-09-11T08:06:41.614826+0000","last_became_peered":"2024-09-11T08:06:41.614826+0000","last_unstale":"2024-09-11T08:06:53.355788+0000","last_undegraded":"2024-09-11T08:06:53.355788+0000","last_fullsized":"2024-09-11T08:06:53.355788+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_clean_scrub_stamp":"2024-09-11T08:06:40.591817+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:15:35.543628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.922718+0000","last_change":"2024-09-11T08:06:43.628775+0000","last_active":"2024-09-11T08:06:53.922718+0000","last_peered":"2024-09-11T08:06:53.922718+0000","last_clean":"2024-09-11T08:06:53.922718+0000","last_became_active":"2024-09-11T08:06:43.628471+0000","last_became_peered":"2024-09-11T08:06:43.628471+0000","last_unstale":"2024-09-11T08:06:53.922718+0000","last_undegraded":"2024-09-11T08:06:53.922718+0000","last_fullsized":"2024-09-11T08:06:53.922718+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_clean_scrub_stamp":"2024-09-11T08:06:42.602559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:50:17.482273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.355972+0000","last_change":"2024-09-11T08:06:45.626960+0000","last_active":"2024-09-11T08:06:53.355972+0000","last_peered":"2024-09-11T08:06:53.355972+0000","last_clean":"2024-09-11T08:06:53.355972+0000","last_became_active":"2024-09-11T08:06:45.626701+0000","last_became_peered":"2024-09-11T08:06:45.626701+0000","last_unstale":"2024-09-11T08:06:53.355972+0000","last_undegraded":"2024-09-11T08:06:53.355972+0000","last_fullsized":"2024-09-11T08:06:53.355972+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_clean_scrub_stamp":"2024-09-11T08:06:44.606510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:51:52.439955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.923048+0000","last_change":"2024-09-11T08:06:39.608517+0000","last_active":"2024-09-11T08:06:53.923048+0000","last_peered":"2024-09-11T08:06:53.923048+0000","last_clean":"2024-09-11T08:06:53.923048+0000","last_became_active":"2024-09-11T08:06:39.608077+0000","last_became_peered":"2024-09-11T08:06:39.608077+0000","last_unstale":"2024-09-11T08:06:53.923048+0000","last_undegraded":"2024-09-11T08:06:53.923048+0000","last_fullsized":"2024-09-11T08:06:53.923048+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_clean_scrub_stamp":"2024-09-11T08:06:38.581279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:43:30.293096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"59'5","reported_seq":26,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.356012+0000","last_change":"2024-09-11T08:06:41.620396+0000","last_active":"2024-09-11T08:06:53.356012+0000","last_peered":"2024-09-11T08:06:53.356012+0000","last_clean":"2024-09-11T08:06:53.356012+0000","last_became_active":"2024-09-11T08:06:41.619929+0000","last_became_peered":"2024-09-11T08:06:41.619929+0000","last_unstale":"2024-09-11T08:06:53.356012+0000","last_undegraded":"2024-09-11T08:06:53.356012+0000","last_fullsized":"2024-09-11T08:06:53.356012+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_clean_scrub_stamp":"2024-09-11T08:06:40.591817+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:10:55.772088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.922331+0000","last_change":"2024-09-11T08:06:43.624549+0000","last_active":"2024-09-11T08:06:53.922331+0000","last_peered":"2024-09-11T08:06:53.922331+0000","last_clean":"2024-09-11T08:06:53.922331+0000","last_became_active":"2024-09-11T08:06:43.624269+0000","last_became_peered":"2024-09-11T08:06:43.624269+0000","last_unstale":"2024-09-11T08:06:53.922331+0000","last_undegraded":"2024-09-11T08:06:53.922331+0000","last_fullsized":"2024-09-11T08:06:53.922331+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_clean_scrub_stamp":"2024-09-11T08:06:42.602559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:46.845168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.921614+0000","last_change":"2024-09-11T08:06:45.626788+0000","last_active":"2024-09-11T08:06:53.921614+0000","last_peered":"2024-09-11T08:06:53.921614+0000","last_clean":"2024-09-11T08:06:53.921614+0000","last_became_active":"2024-09-11T08:06:45.626296+0000","last_became_peered":"2024-09-11T08:06:45.626296+0000","last_unstale":"2024-09-11T08:06:53.921614+0000","last_undegraded":"2024-09-11T08:06:53.921614+0000","last_fullsized":"2024-09-11T08:06:53.921614+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_clean_scrub_stamp":"2024-09-11T08:06:44.606510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:18:33.095602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.922675+0000","last_change":"2024-09-11T08:06:39.602077+0000","last_active":"2024-09-11T08:06:53.922675+0000","last_peered":"2024-09-11T08:06:53.922675+0000","last_clean":"2024-09-11T08:06:53.922675+0000","last_became_active":"2024-09-11T08:06:39.601816+0000","last_became_peered":"2024-09-11T08:06:39.601816+0000","last_unstale":"2024-09-11T08:06:53.922675+0000","last_undegraded":"2024-09-11T08:06:53.922675+0000","last_fullsized":"2024-09-11T08:06:53.922675+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_clean_scrub_stamp":"2024-09-11T08:06:38.581279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:22:17.763220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"59'9","reported_seq":32,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.356154+0000","last_change":"2024-09-11T08:06:41.613120+0000","last_active":"2024-09-11T08:06:53.356154+0000","last_peered":"2024-09-11T08:06:53.356154+0000","last_clean":"2024-09-11T08:06:53.356154+0000","last_became_active":"2024-09-11T08:06:41.612829+0000","last_became_peered":"2024-09-11T08:06:41.612829+0000","last_unstale":"2024-09-11T08:06:53.356154+0000","last_undegraded":"2024-09-11T08:06:53.356154+0000","last_fullsized":"2024-09-11T08:06:53.356154+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_clean_scrub_stamp":"2024-09-11T08:06:40.591817+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:18:05.795092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.923237+0000","last_change":"2024-09-11T08:06:43.627062+0000","last_active":"2024-09-11T08:06:53.923237+0000","last_peered":"2024-09-11T08:06:53.923237+0000","last_clean":"2024-09-11T08:06:53.923237+0000","last_became_active":"2024-09-11T08:06:43.626763+0000","last_became_peered":"2024-09-11T08:06:43.626763+0000","last_unstale":"2024-09-11T08:06:53.923237+0000","last_undegraded":"2024-09-11T08:06:53.923237+0000","last_fullsized":"2024-09-11T08:06:53.923237+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_clean_scrub_stamp":"2024-09-11T08:06:42.602559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:19:00.953811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.357093+0000","last_change":"2024-09-11T08:06:45.627878+0000","last_active":"2024-09-11T08:06:53.357093+0000","last_peered":"2024-09-11T08:06:53.357093+0000","last_clean":"2024-09-11T08:06:53.357093+0000","last_became_active":"2024-09-11T08:06:45.627608+0000","last_became_peered":"2024-09-11T08:06:45.627608+0000","last_unstale":"2024-09-11T08:06:53.357093+0000","last_undegraded":"2024-09-11T08:06:53.357093+0000","last_fullsized":"2024-09-11T08:06:53.357093+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_clean_scrub_stamp":"2024-09-11T08:06:44.606510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:40:58.490513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.d","version":"59'17","reported_seq":44,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.356350+0000","last_change":"2024-09-11T08:06:41.620451+0000","last_active":"2024-09-11T08:06:53.356350+0000","last_peered":"2024-09-11T08:06:53.356350+0000","last_clean":"2024-09-11T08:06:53.356350+0000","last_became_active":"2024-09-11T08:06:41.620134+0000","last_became_peered":"2024-09-11T08:06:41.620134+0000","last_unstale":"2024-09-11T08:06:53.356350+0000","last_undegraded":"2024-09-11T08:06:53.356350+0000","last_fullsized":"2024-09-11T08:06:53.356350+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_clean_scrub_stamp":"2024-09-11T08:06:40.591817+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:18:41.188706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.922343+0000","last_change":"2024-09-11T08:06:39.603012+0000","last_active":"2024-09-11T08:06:53.922343+0000","last_peered":"2024-09-11T08:06:53.922343+0000","last_clean":"2024-09-11T08:06:53.922343+0000","last_became_active":"2024-09-11T08:06:39.602726+0000","last_became_peered":"2024-09-11T08:06:39.602726+0000","last_unstale":"2024-09-11T08:06:53.922343+0000","last_undegraded":"2024-09-11T08:06:53.922343+0000","last_fullsized":"2024-09-11T08:06:53.922343+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_clean_scrub_stamp":"2024-09-11T08:06:38.581279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:33:33.441740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.922915+0000","last_change":"2024-09-11T08:06:43.626619+0000","last_active":"2024-09-11T08:06:53.922915+0000","last_peered":"2024-09-11T08:06:53.922915+0000","last_clean":"2024-09-11T08:06:53.922915+0000","last_became_active":"2024-09-11T08:06:43.626348+0000","last_became_peered":"2024-09-11T08:06:43.626348+0000","last_unstale":"2024-09-11T08:06:53.922915+0000","last_undegraded":"2024-09-11T08:06:53.922915+0000","last_fullsized":"2024-09-11T08:06:53.922915+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_clean_scrub_stamp":"2024-09-11T08:06:42.602559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:13:51.077967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.358220+0000","last_change":"2024-09-11T08:06:45.624937+0000","last_active":"2024-09-11T08:06:53.358220+0000","last_peered":"2024-09-11T08:06:53.358220+0000","last_clean":"2024-09-11T08:06:53.358220+0000","last_became_active":"2024-09-11T08:06:45.624654+0000","last_became_peered":"2024-09-11T08:06:45.624654+0000","last_unstale":"2024-09-11T08:06:53.358220+0000","last_undegraded":"2024-09-11T08:06:53.358220+0000","last_fullsized":"2024-09-11T08:06:53.358220+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_clean_scrub_stamp":"2024-09-11T08:06:44.606510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:52:35.974581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"59'10","reported_seq":31,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.356179+0000","last_change":"2024-09-11T08:06:41.612868+0000","last_active":"2024-09-11T08:06:53.356179+0000","last_peered":"2024-09-11T08:06:53.356179+0000","last_clean":"2024-09-11T08:06:53.356179+0000","last_became_active":"2024-09-11T08:06:41.612541+0000","last_became_peered":"2024-09-11T08:06:41.612541+0000","last_unstale":"2024-09-11T08:06:53.356179+0000","last_undegraded":"2024-09-11T08:06:53.356179+0000","last_fullsized":"2024-09-11T08:06:53.356179+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_clean_scrub_stamp":"2024-09-11T08:06:40.591817+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:04:23.460925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.923245+0000","last_change":"2024-09-11T08:06:39.605945+0000","last_active":"2024-09-11T08:06:53.923245+0000","last_peered":"2024-09-11T08:06:53.923245+0000","last_clean":"2024-09-11T08:06:53.923245+0000","last_became_active":"2024-09-11T08:06:39.605672+0000","last_became_peered":"2024-09-11T08:06:39.605672+0000","last_unstale":"2024-09-11T08:06:53.923245+0000","last_undegraded":"2024-09-11T08:06:53.923245+0000","last_fullsized":"2024-09-11T08:06:53.923245+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_clean_scrub_stamp":"2024-09-11T08:06:38.581279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:35:07.469717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"59'8","reported_seq":38,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:07:53.122866+0000","last_change":"2024-09-11T08:06:43.628373+0000","last_active":"2024-09-11T08:07:53.122866+0000","last_peered":"2024-09-11T08:07:53.122866+0000","last_clean":"2024-09-11T08:07:53.122866+0000","last_became_active":"2024-09-11T08:06:43.628087+0000","last_became_peered":"2024-09-11T08:06:43.628087+0000","last_unstale":"2024-09-11T08:07:53.122866+0000","last_undegraded":"2024-09-11T08:07:53.122866+0000","last_fullsized":"2024-09-11T08:07:53.122866+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_clean_scrub_stamp":"2024-09-11T08:06:42.602559+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:47:05.748888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.356309+0000","last_change":"2024-09-11T08:06:45.627617+0000","last_active":"2024-09-11T08:06:53.356309+0000","last_peered":"2024-09-11T08:06:53.356309+0000","last_clean":"2024-09-11T08:06:53.356309+0000","last_became_active":"2024-09-11T08:06:45.626825+0000","last_became_peered":"2024-09-11T08:06:45.626825+0000","last_unstale":"2024-09-11T08:06:53.356309+0000","last_undegraded":"2024-09-11T08:06:53.356309+0000","last_fullsized":"2024-09-11T08:06:53.356309+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_clean_scrub_stamp":"2024-09-11T08:06:44.606510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:02:05.584271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"59'15","reported_seq":41,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.923558+0000","last_change":"2024-09-11T08:06:41.619820+0000","last_active":"2024-09-11T08:06:53.923558+0000","last_peered":"2024-09-11T08:06:53.923558+0000","last_clean":"2024-09-11T08:06:53.923558+0000","last_became_active":"2024-09-11T08:06:41.619534+0000","last_became_peered":"2024-09-11T08:06:41.619534+0000","last_unstale":"2024-09-11T08:06:53.923558+0000","last_undegraded":"2024-09-11T08:06:53.923558+0000","last_fullsized":"2024-09-11T08:06:53.923558+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_clean_scrub_stamp":"2024-09-11T08:06:40.591817+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:40:30.644923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.922840+0000","last_change":"2024-09-11T08:06:39.608392+0000","last_active":"2024-09-11T08:06:53.922840+0000","last_peered":"2024-09-11T08:06:53.922840+0000","last_clean":"2024-09-11T08:06:53.922840+0000","last_became_active":"2024-09-11T08:06:39.608060+0000","last_became_peered":"2024-09-11T08:06:39.608060+0000","last_unstale":"2024-09-11T08:06:53.922840+0000","last_undegraded":"2024-09-11T08:06:53.922840+0000","last_fullsized":"2024-09-11T08:06:53.922840+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_clean_scrub_stamp":"2024-09-11T08:06:38.581279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:14:47.084168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"59'8","reported_seq":38,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:07:53.123125+0000","last_change":"2024-09-11T08:06:43.629899+0000","last_active":"2024-09-11T08:07:53.123125+0000","last_peered":"2024-09-11T08:07:53.123125+0000","last_clean":"2024-09-11T08:07:53.123125+0000","last_became_active":"2024-09-11T08:06:43.629636+0000","last_became_peered":"2024-09-11T08:06:43.629636+0000","last_unstale":"2024-09-11T08:07:53.123125+0000","last_undegraded":"2024-09-11T08:07:53.123125+0000","last_fullsized":"2024-09-11T08:07:53.123125+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_clean_scrub_stamp":"2024-09-11T08:06:42.602559+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:23:53.155784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.923667+0000","last_change":"2024-09-11T08:06:45.632863+0000","last_active":"2024-09-11T08:06:53.923667+0000","last_peered":"2024-09-11T08:06:53.923667+0000","last_clean":"2024-09-11T08:06:53.923667+0000","last_became_active":"2024-09-11T08:06:45.632518+0000","last_became_peered":"2024-09-11T08:06:45.632518+0000","last_unstale":"2024-09-11T08:06:53.923667+0000","last_undegraded":"2024-09-11T08:06:53.923667+0000","last_fullsized":"2024-09-11T08:06:53.923667+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_clean_scrub_stamp":"2024-09-11T08:06:44.606510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:50:43.840099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"59'11","reported_seq":35,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.356459+0000","last_change":"2024-09-11T08:06:41.620101+0000","last_active":"2024-09-11T08:06:53.356459+0000","last_peered":"2024-09-11T08:06:53.356459+0000","last_clean":"2024-09-11T08:06:53.356459+0000","last_became_active":"2024-09-11T08:06:41.619521+0000","last_became_peered":"2024-09-11T08:06:41.619521+0000","last_unstale":"2024-09-11T08:06:53.356459+0000","last_undegraded":"2024-09-11T08:06:53.356459+0000","last_fullsized":"2024-09-11T08:06:53.356459+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_clean_scrub_stamp":"2024-09-11T08:06:40.591817+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:12:28.192461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.922976+0000","last_change":"2024-09-11T08:06:39.609950+0000","last_active":"2024-09-11T08:06:53.922976+0000","last_peered":"2024-09-11T08:06:53.922976+0000","last_clean":"2024-09-11T08:06:53.922976+0000","last_became_active":"2024-09-11T08:06:39.609215+0000","last_became_peered":"2024-09-11T08:06:39.609215+0000","last_unstale":"2024-09-11T08:06:53.922976+0000","last_undegraded":"2024-09-11T08:06:53.922976+0000","last_fullsized":"2024-09-11T08:06:53.922976+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_clean_scrub_stamp":"2024-09-11T08:06:38.581279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:07:31.708252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.923809+0000","last_change":"2024-09-11T08:06:43.627822+0000","last_active":"2024-09-11T08:06:53.923809+0000","last_peered":"2024-09-11T08:06:53.923809+0000","last_clean":"2024-09-11T08:06:53.923809+0000","last_became_active":"2024-09-11T08:06:43.627558+0000","last_became_peered":"2024-09-11T08:06:43.627558+0000","last_unstale":"2024-09-11T08:06:53.923809+0000","last_undegraded":"2024-09-11T08:06:53.923809+0000","last_fullsized":"2024-09-11T08:06:53.923809+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_clean_scrub_stamp":"2024-09-11T08:06:42.602559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:50:09.567644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.505178+0000","last_change":"2024-09-11T08:06:46.172869+0000","last_active":"2024-09-11T08:06:53.505178+0000","last_peered":"2024-09-11T08:06:53.505178+0000","last_clean":"2024-09-11T08:06:53.505178+0000","last_became_active":"2024-09-11T08:06:46.172458+0000","last_became_peered":"2024-09-11T08:06:46.172458+0000","last_unstale":"2024-09-11T08:06:53.505178+0000","last_undegraded":"2024-09-11T08:06:53.505178+0000","last_fullsized":"2024-09-11T08:06:53.505178+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_clean_scrub_stamp":"2024-09-11T08:06:44.606510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:10:31.229111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"59'14","reported_seq":37,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.356071+0000","last_change":"2024-09-11T08:06:41.613763+0000","last_active":"2024-09-11T08:06:53.356071+0000","last_peered":"2024-09-11T08:06:53.356071+0000","last_clean":"2024-09-11T08:06:53.356071+0000","last_became_active":"2024-09-11T08:06:41.613474+0000","last_became_peered":"2024-09-11T08:06:41.613474+0000","last_unstale":"2024-09-11T08:06:53.356071+0000","last_undegraded":"2024-09-11T08:06:53.356071+0000","last_fullsized":"2024-09-11T08:06:53.356071+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_clean_scrub_stamp":"2024-09-11T08:06:40.591817+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:08:59.356386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.922931+0000","last_change":"2024-09-11T08:06:39.603231+0000","last_active":"2024-09-11T08:06:53.922931+0000","last_peered":"2024-09-11T08:06:53.922931+0000","last_clean":"2024-09-11T08:06:53.922931+0000","last_became_active":"2024-09-11T08:06:39.602503+0000","last_became_peered":"2024-09-11T08:06:39.602503+0000","last_unstale":"2024-09-11T08:06:53.922931+0000","last_undegraded":"2024-09-11T08:06:53.922931+0000","last_fullsized":"2024-09-11T08:06:53.922931+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_clean_scrub_stamp":"2024-09-11T08:06:38.581279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:16:47.034701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"59'8","reported_seq":41,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:07:53.122923+0000","last_change":"2024-09-11T08:06:43.627128+0000","last_active":"2024-09-11T08:07:53.122923+0000","last_peered":"2024-09-11T08:07:53.122923+0000","last_clean":"2024-09-11T08:07:53.122923+0000","last_became_active":"2024-09-11T08:06:43.626849+0000","last_became_peered":"2024-09-11T08:06:43.626849+0000","last_unstale":"2024-09-11T08:07:53.122923+0000","last_undegraded":"2024-09-11T08:07:53.122923+0000","last_fullsized":"2024-09-11T08:07:53.122923+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_clean_scrub_stamp":"2024-09-11T08:06:42.602559+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:24:27.021095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.922203+0000","last_change":"2024-09-11T08:06:46.175440+0000","last_active":"2024-09-11T08:06:53.922203+0000","last_peered":"2024-09-11T08:06:53.922203+0000","last_clean":"2024-09-11T08:06:53.922203+0000","last_became_active":"2024-09-11T08:06:46.175157+0000","last_became_peered":"2024-09-11T08:06:46.175157+0000","last_unstale":"2024-09-11T08:06:53.922203+0000","last_undegraded":"2024-09-11T08:06:53.922203+0000","last_fullsized":"2024-09-11T08:06:53.922203+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_clean_scrub_stamp":"2024-09-11T08:06:44.606510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:49:16.423291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"59'18","reported_seq":48,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.504760+0000","last_change":"2024-09-11T08:06:42.465901+0000","last_active":"2024-09-11T08:06:53.504760+0000","last_peered":"2024-09-11T08:06:53.504760+0000","last_clean":"2024-09-11T08:06:53.504760+0000","last_became_active":"2024-09-11T08:06:42.465624+0000","last_became_peered":"2024-09-11T08:06:42.465624+0000","last_unstale":"2024-09-11T08:06:53.504760+0000","last_undegraded":"2024-09-11T08:06:53.504760+0000","last_fullsized":"2024-09-11T08:06:53.504760+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_clean_scrub_stamp":"2024-09-11T08:06:40.591817+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:37:31.078992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.923650+0000","last_change":"2024-09-11T08:06:39.610426+0000","last_active":"2024-09-11T08:06:53.923650+0000","last_peered":"2024-09-11T08:06:53.923650+0000","last_clean":"2024-09-11T08:06:53.923650+0000","last_became_active":"2024-09-11T08:06:39.610142+0000","last_became_peered":"2024-09-11T08:06:39.610142+0000","last_unstale":"2024-09-11T08:06:53.923650+0000","last_undegraded":"2024-09-11T08:06:53.923650+0000","last_fullsized":"2024-09-11T08:06:53.923650+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_clean_scrub_stamp":"2024-09-11T08:06:38.581279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:00.085742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.355863+0000","last_change":"2024-09-11T08:06:43.621918+0000","last_active":"2024-09-11T08:06:53.355863+0000","last_peered":"2024-09-11T08:06:53.355863+0000","last_clean":"2024-09-11T08:06:53.355863+0000","last_became_active":"2024-09-11T08:06:43.620966+0000","last_became_peered":"2024-09-11T08:06:43.620966+0000","last_unstale":"2024-09-11T08:06:53.355863+0000","last_undegraded":"2024-09-11T08:06:53.355863+0000","last_fullsized":"2024-09-11T08:06:53.355863+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_clean_scrub_stamp":"2024-09-11T08:06:42.602559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:28:54.651591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.356153+0000","last_change":"2024-09-11T08:06:45.628163+0000","last_active":"2024-09-11T08:06:53.356153+0000","last_peered":"2024-09-11T08:06:53.356153+0000","last_clean":"2024-09-11T08:06:53.356153+0000","last_became_active":"2024-09-11T08:06:45.627725+0000","last_became_peered":"2024-09-11T08:06:45.627725+0000","last_unstale":"2024-09-11T08:06:53.356153+0000","last_undegraded":"2024-09-11T08:06:53.356153+0000","last_fullsized":"2024-09-11T08:06:53.356153+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_clean_scrub_stamp":"2024-09-11T08:06:44.606510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:14:54.778445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"59'19","reported_seq":52,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.357887+0000","last_change":"2024-09-11T08:06:42.463317+0000","last_active":"2024-09-11T08:06:53.357887+0000","last_peered":"2024-09-11T08:06:53.357887+0000","last_clean":"2024-09-11T08:06:53.357887+0000","last_became_active":"2024-09-11T08:06:42.462993+0000","last_became_peered":"2024-09-11T08:06:42.462993+0000","last_unstale":"2024-09-11T08:06:53.357887+0000","last_undegraded":"2024-09-11T08:06:53.357887+0000","last_fullsized":"2024-09-11T08:06:53.357887+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_clean_scrub_stamp":"2024-09-11T08:06:40.591817+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:21:04.443228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.923073+0000","last_change":"2024-09-11T08:06:39.608340+0000","last_active":"2024-09-11T08:06:53.923073+0000","last_peered":"2024-09-11T08:06:53.923073+0000","last_clean":"2024-09-11T08:06:53.923073+0000","last_became_active":"2024-09-11T08:06:39.607713+0000","last_became_peered":"2024-09-11T08:06:39.607713+0000","last_unstale":"2024-09-11T08:06:53.923073+0000","last_undegraded":"2024-09-11T08:06:53.923073+0000","last_fullsized":"2024-09-11T08:06:53.923073+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_clean_scrub_stamp":"2024-09-11T08:06:38.581279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:53:46.241083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.922610+0000","last_change":"2024-09-11T08:06:43.627376+0000","last_active":"2024-09-11T08:06:53.922610+0000","last_peered":"2024-09-11T08:06:53.922610+0000","last_clean":"2024-09-11T08:06:53.922610+0000","last_became_active":"2024-09-11T08:06:43.627099+0000","last_became_peered":"2024-09-11T08:06:43.627099+0000","last_unstale":"2024-09-11T08:06:53.922610+0000","last_undegraded":"2024-09-11T08:06:53.922610+0000","last_fullsized":"2024-09-11T08:06:53.922610+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_clean_scrub_stamp":"2024-09-11T08:06:42.602559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:06:03.180648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.922471+0000","last_change":"2024-09-11T08:06:46.172840+0000","last_active":"2024-09-11T08:06:53.922471+0000","last_peered":"2024-09-11T08:06:53.922471+0000","last_clean":"2024-09-11T08:06:53.922471+0000","last_became_active":"2024-09-11T08:06:46.172430+0000","last_became_peered":"2024-09-11T08:06:46.172430+0000","last_unstale":"2024-09-11T08:06:53.922471+0000","last_undegraded":"2024-09-11T08:06:53.922471+0000","last_fullsized":"2024-09-11T08:06:53.922471+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_clean_scrub_stamp":"2024-09-11T08:06:44.606510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:46:46.231346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"59'10","reported_seq":31,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.923176+0000","last_change":"2024-09-11T08:06:41.618705+0000","last_active":"2024-09-11T08:06:53.923176+0000","last_peered":"2024-09-11T08:06:53.923176+0000","last_clean":"2024-09-11T08:06:53.923176+0000","last_became_active":"2024-09-11T08:06:41.617495+0000","last_became_peered":"2024-09-11T08:06:41.617495+0000","last_unstale":"2024-09-11T08:06:53.923176+0000","last_undegraded":"2024-09-11T08:06:53.923176+0000","last_fullsized":"2024-09-11T08:06:53.923176+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_clean_scrub_stamp":"2024-09-11T08:06:40.591817+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:24:32.460028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.923109+0000","last_change":"2024-09-11T08:06:39.606512+0000","last_active":"2024-09-11T08:06:53.923109+0000","last_peered":"2024-09-11T08:06:53.923109+0000","last_clean":"2024-09-11T08:06:53.923109+0000","last_became_active":"2024-09-11T08:06:39.606234+0000","last_became_peered":"2024-09-11T08:06:39.606234+0000","last_unstale":"2024-09-11T08:06:53.923109+0000","last_undegraded":"2024-09-11T08:06:53.923109+0000","last_fullsized":"2024-09-11T08:06:53.923109+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_clean_scrub_stamp":"2024-09-11T08:06:38.581279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:35:51.447680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"59'8","reported_seq":38,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:07:53.122872+0000","last_change":"2024-09-11T08:06:43.630234+0000","last_active":"2024-09-11T08:07:53.122872+0000","last_peered":"2024-09-11T08:07:53.122872+0000","last_clean":"2024-09-11T08:07:53.122872+0000","last_became_active":"2024-09-11T08:06:43.629808+0000","last_became_peered":"2024-09-11T08:06:43.629808+0000","last_unstale":"2024-09-11T08:07:53.122872+0000","last_undegraded":"2024-09-11T08:07:53.122872+0000","last_fullsized":"2024-09-11T08:07:53.122872+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_clean_scrub_stamp":"2024-09-11T08:06:42.602559+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:37:08.879992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.356941+0000","last_change":"2024-09-11T08:06:45.624297+0000","last_active":"2024-09-11T08:06:53.356941+0000","last_peered":"2024-09-11T08:06:53.356941+0000","last_clean":"2024-09-11T08:06:53.356941+0000","last_became_active":"2024-09-11T08:06:45.624005+0000","last_became_peered":"2024-09-11T08:06:45.624005+0000","last_unstale":"2024-09-11T08:06:53.356941+0000","last_undegraded":"2024-09-11T08:06:53.356941+0000","last_fullsized":"2024-09-11T08:06:53.356941+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_clean_scrub_stamp":"2024-09-11T08:06:44.606510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:27:21.190292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"59'28","reported_seq":80,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:07:53.122849+0000","last_change":"2024-09-11T08:06:41.622452+0000","last_active":"2024-09-11T08:07:53.122849+0000","last_peered":"2024-09-11T08:07:53.122849+0000","last_clean":"2024-09-11T08:07:53.122849+0000","last_became_active":"2024-09-11T08:06:41.622118+0000","last_became_peered":"2024-09-11T08:06:41.622118+0000","last_unstale":"2024-09-11T08:07:53.122849+0000","last_undegraded":"2024-09-11T08:07:53.122849+0000","last_fullsized":"2024-09-11T08:07:53.122849+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_clean_scrub_stamp":"2024-09-11T08:06:40.591817+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:15:46.142935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.923538+0000","last_change":"2024-09-11T08:06:39.604374+0000","last_active":"2024-09-11T08:06:53.923538+0000","last_peered":"2024-09-11T08:06:53.923538+0000","last_clean":"2024-09-11T08:06:53.923538+0000","last_became_active":"2024-09-11T08:06:39.604076+0000","last_became_peered":"2024-09-11T08:06:39.604076+0000","last_unstale":"2024-09-11T08:06:53.923538+0000","last_undegraded":"2024-09-11T08:06:53.923538+0000","last_fullsized":"2024-09-11T08:06:53.923538+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_clean_scrub_stamp":"2024-09-11T08:06:38.581279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:05:34.368960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.357382+0000","last_change":"2024-09-11T08:06:43.619964+0000","last_active":"2024-09-11T08:06:53.357382+0000","last_peered":"2024-09-11T08:06:53.357382+0000","last_clean":"2024-09-11T08:06:53.357382+0000","last_became_active":"2024-09-11T08:06:43.619685+0000","last_became_peered":"2024-09-11T08:06:43.619685+0000","last_unstale":"2024-09-11T08:06:53.357382+0000","last_undegraded":"2024-09-11T08:06:53.357382+0000","last_fullsized":"2024-09-11T08:06:53.357382+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_clean_scrub_stamp":"2024-09-11T08:06:42.602559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:14:41.595586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.505236+0000","last_change":"2024-09-11T08:06:45.627245+0000","last_active":"2024-09-11T08:06:53.505236+0000","last_peered":"2024-09-11T08:06:53.505236+0000","last_clean":"2024-09-11T08:06:53.505236+0000","last_became_active":"2024-09-11T08:06:45.626919+0000","last_became_peered":"2024-09-11T08:06:45.626919+0000","last_unstale":"2024-09-11T08:06:53.505236+0000","last_undegraded":"2024-09-11T08:06:53.505236+0000","last_fullsized":"2024-09-11T08:06:53.505236+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_clean_scrub_stamp":"2024-09-11T08:06:44.606510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:59:15.307403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"59'13","reported_seq":43,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.923622+0000","last_change":"2024-09-11T08:06:41.618883+0000","last_active":"2024-09-11T08:06:53.923622+0000","last_peered":"2024-09-11T08:06:53.923622+0000","last_clean":"2024-09-11T08:06:53.923622+0000","last_became_active":"2024-09-11T08:06:41.618616+0000","last_became_peered":"2024-09-11T08:06:41.618616+0000","last_unstale":"2024-09-11T08:06:53.923622+0000","last_undegraded":"2024-09-11T08:06:53.923622+0000","last_fullsized":"2024-09-11T08:06:53.923622+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_clean_scrub_stamp":"2024-09-11T08:06:40.591817+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:09:02.784263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"59'5","reported_seq":103,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:08:02.044505+0000","last_change":"2024-09-11T08:06:43.619846+0000","last_active":"2024-09-11T08:08:02.044505+0000","last_peered":"2024-09-11T08:08:02.044505+0000","last_clean":"2024-09-11T08:08:02.044505+0000","last_became_active":"2024-09-11T08:06:39.609471+0000","last_became_peered":"2024-09-11T08:06:39.609471+0000","last_unstale":"2024-09-11T08:08:02.044505+0000","last_undegraded":"2024-09-11T08:08:02.044505+0000","last_fullsized":"2024-09-11T08:08:02.044505+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_clean_scrub_stamp":"2024-09-11T08:06:38.581279+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:00:36.163114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056896900000000001,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":77,"num_read_kb":72,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.357000+0000","last_change":"2024-09-11T08:06:39.606585+0000","last_active":"2024-09-11T08:06:53.357000+0000","last_peered":"2024-09-11T08:06:53.357000+0000","last_clean":"2024-09-11T08:06:53.357000+0000","last_became_active":"2024-09-11T08:06:39.606331+0000","last_became_peered":"2024-09-11T08:06:39.606331+0000","last_unstale":"2024-09-11T08:06:53.357000+0000","last_undegraded":"2024-09-11T08:06:53.357000+0000","last_fullsized":"2024-09-11T08:06:53.357000+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_clean_scrub_stamp":"2024-09-11T08:06:38.581279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:21:21.466087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.357889+0000","last_change":"2024-09-11T08:06:43.629359+0000","last_active":"2024-09-11T08:06:53.357889+0000","last_peered":"2024-09-11T08:06:53.357889+0000","last_clean":"2024-09-11T08:06:53.357889+0000","last_became_active":"2024-09-11T08:06:43.629097+0000","last_became_peered":"2024-09-11T08:06:43.629097+0000","last_unstale":"2024-09-11T08:06:53.357889+0000","last_undegraded":"2024-09-11T08:06:53.357889+0000","last_fullsized":"2024-09-11T08:06:53.357889+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_clean_scrub_stamp":"2024-09-11T08:06:42.602559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:22:55.100376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.922546+0000","last_change":"2024-09-11T08:06:46.175308+0000","last_active":"2024-09-11T08:06:53.922546+0000","last_peered":"2024-09-11T08:06:53.922546+0000","last_clean":"2024-09-11T08:06:53.922546+0000","last_became_active":"2024-09-11T08:06:46.175011+0000","last_became_peered":"2024-09-11T08:06:46.175011+0000","last_unstale":"2024-09-11T08:06:53.922546+0000","last_undegraded":"2024-09-11T08:06:53.922546+0000","last_fullsized":"2024-09-11T08:06:53.922546+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_clean_scrub_stamp":"2024-09-11T08:06:44.606510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:00:43.461643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"59'12","reported_seq":34,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.358562+0000","last_change":"2024-09-11T08:06:41.616886+0000","last_active":"2024-09-11T08:06:53.358562+0000","last_peered":"2024-09-11T08:06:53.358562+0000","last_clean":"2024-09-11T08:06:53.358562+0000","last_became_active":"2024-09-11T08:06:41.616634+0000","last_became_peered":"2024-09-11T08:06:41.616634+0000","last_unstale":"2024-09-11T08:06:53.358562+0000","last_undegraded":"2024-09-11T08:06:53.358562+0000","last_fullsized":"2024-09-11T08:06:53.358562+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_clean_scrub_stamp":"2024-09-11T08:06:40.591817+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:52:19.243060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"54'1","reported_seq":24,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.358142+0000","last_change":"2024-09-11T08:06:43.610415+0000","last_active":"2024-09-11T08:06:53.358142+0000","last_peered":"2024-09-11T08:06:53.358142+0000","last_clean":"2024-09-11T08:06:53.358142+0000","last_became_active":"2024-09-11T08:06:39.607546+0000","last_became_peered":"2024-09-11T08:06:39.607546+0000","last_unstale":"2024-09-11T08:06:53.358142+0000","last_undegraded":"2024-09-11T08:06:53.358142+0000","last_fullsized":"2024-09-11T08:06:53.358142+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_clean_scrub_stamp":"2024-09-11T08:06:38.581279+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:47:48.094686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000423239,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"52'1","reported_seq":25,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.923066+0000","last_change":"2024-09-11T08:06:39.609049+0000","last_active":"2024-09-11T08:06:53.923066+0000","last_peered":"2024-09-11T08:06:53.923066+0000","last_clean":"2024-09-11T08:06:53.923066+0000","last_became_active":"2024-09-11T08:06:39.608473+0000","last_became_peered":"2024-09-11T08:06:39.608473+0000","last_unstale":"2024-09-11T08:06:53.923066+0000","last_undegraded":"2024-09-11T08:06:53.923066+0000","last_fullsized":"2024-09-11T08:06:53.923066+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_clean_scrub_stamp":"2024-09-11T08:06:38.581279+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:47:06.604637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.923877+0000","last_change":"2024-09-11T08:06:43.627680+0000","last_active":"2024-09-11T08:06:53.923877+0000","last_peered":"2024-09-11T08:06:53.923877+0000","last_clean":"2024-09-11T08:06:53.923877+0000","last_became_active":"2024-09-11T08:06:43.627405+0000","last_became_peered":"2024-09-11T08:06:43.627405+0000","last_unstale":"2024-09-11T08:06:53.923877+0000","last_undegraded":"2024-09-11T08:06:53.923877+0000","last_fullsized":"2024-09-11T08:06:53.923877+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_clean_scrub_stamp":"2024-09-11T08:06:42.602559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:56:12.256792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.922327+0000","last_change":"2024-09-11T08:06:45.626461+0000","last_active":"2024-09-11T08:06:53.922327+0000","last_peered":"2024-09-11T08:06:53.922327+0000","last_clean":"2024-09-11T08:06:53.922327+0000","last_became_active":"2024-09-11T08:06:45.626124+0000","last_became_peered":"2024-09-11T08:06:45.626124+0000","last_unstale":"2024-09-11T08:06:53.922327+0000","last_undegraded":"2024-09-11T08:06:53.922327+0000","last_fullsized":"2024-09-11T08:06:53.922327+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_clean_scrub_stamp":"2024-09-11T08:06:44.606510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:38:52.071915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"59'14","reported_seq":38,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.922489+0000","last_change":"2024-09-11T08:06:42.464026+0000","last_active":"2024-09-11T08:06:53.922489+0000","last_peered":"2024-09-11T08:06:53.922489+0000","last_clean":"2024-09-11T08:06:53.922489+0000","last_became_active":"2024-09-11T08:06:42.463593+0000","last_became_peered":"2024-09-11T08:06:42.463593+0000","last_unstale":"2024-09-11T08:06:53.922489+0000","last_undegraded":"2024-09-11T08:06:53.922489+0000","last_fullsized":"2024-09-11T08:06:53.922489+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_clean_scrub_stamp":"2024-09-11T08:06:40.591817+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:56:48.344165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"56'2","reported_seq":25,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.505088+0000","last_change":"2024-09-11T08:06:43.617993+0000","last_active":"2024-09-11T08:06:53.505088+0000","last_peered":"2024-09-11T08:06:53.505088+0000","last_clean":"2024-09-11T08:06:53.505088+0000","last_became_active":"2024-09-11T08:06:39.602072+0000","last_became_peered":"2024-09-11T08:06:39.602072+0000","last_unstale":"2024-09-11T08:06:53.505088+0000","last_undegraded":"2024-09-11T08:06:53.505088+0000","last_fullsized":"2024-09-11T08:06:53.505088+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_clean_scrub_stamp":"2024-09-11T08:06:38.581279+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:23:58.029145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001549904,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.923605+0000","last_change":"2024-09-11T08:06:39.605901+0000","last_active":"2024-09-11T08:06:53.923605+0000","last_peered":"2024-09-11T08:06:53.923605+0000","last_clean":"2024-09-11T08:06:53.923605+0000","last_became_active":"2024-09-11T08:06:39.605647+0000","last_became_peered":"2024-09-11T08:06:39.605647+0000","last_unstale":"2024-09-11T08:06:53.923605+0000","last_undegraded":"2024-09-11T08:06:53.923605+0000","last_fullsized":"2024-09-11T08:06:53.923605+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_clean_scrub_stamp":"2024-09-11T08:06:38.581279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:32:10.154369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"61'79","reported_seq":52,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:07:19.108170+0000","last_change":"2024-09-11T08:06:29.133512+0000","last_active":"2024-09-11T08:07:19.108170+0000","last_peered":"2024-09-11T08:07:19.108170+0000","last_clean":"2024-09-11T08:07:19.108170+0000","last_became_active":"2024-09-11T08:06:29.127939+0000","last_became_peered":"2024-09-11T08:06:29.127939+0000","last_unstale":"2024-09-11T08:07:19.108170+0000","last_undegraded":"2024-09-11T08:07:19.108170+0000","last_fullsized":"2024-09-11T08:07:19.108170+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:04:29.497369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:04:29.497369+0000","last_clean_scrub_stamp":"2024-09-11T08:04:29.497369+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:30:31.230878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.922446+0000","last_change":"2024-09-11T08:06:43.628742+0000","last_active":"2024-09-11T08:06:53.922446+0000","last_peered":"2024-09-11T08:06:53.922446+0000","last_clean":"2024-09-11T08:06:53.922446+0000","last_became_active":"2024-09-11T08:06:43.628466+0000","last_became_peered":"2024-09-11T08:06:43.628466+0000","last_unstale":"2024-09-11T08:06:53.922446+0000","last_undegraded":"2024-09-11T08:06:53.922446+0000","last_fullsized":"2024-09-11T08:06:53.922446+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_clean_scrub_stamp":"2024-09-11T08:06:42.602559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:33:07.821898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.921946+0000","last_change":"2024-09-11T08:06:45.628579+0000","last_active":"2024-09-11T08:06:53.921946+0000","last_peered":"2024-09-11T08:06:53.921946+0000","last_clean":"2024-09-11T08:06:53.921946+0000","last_became_active":"2024-09-11T08:06:45.628293+0000","last_became_peered":"2024-09-11T08:06:45.628293+0000","last_unstale":"2024-09-11T08:06:53.921946+0000","last_undegraded":"2024-09-11T08:06:53.921946+0000","last_fullsized":"2024-09-11T08:06:53.921946+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_clean_scrub_stamp":"2024-09-11T08:06:44.606510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:23:51.069892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"59'19","reported_seq":47,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.922748+0000","last_change":"2024-09-11T08:06:42.464039+0000","last_active":"2024-09-11T08:06:53.922748+0000","last_peered":"2024-09-11T08:06:53.922748+0000","last_clean":"2024-09-11T08:06:53.922748+0000","last_became_active":"2024-09-11T08:06:42.462906+0000","last_became_peered":"2024-09-11T08:06:42.462906+0000","last_unstale":"2024-09-11T08:06:53.922748+0000","last_undegraded":"2024-09-11T08:06:53.922748+0000","last_fullsized":"2024-09-11T08:06:53.922748+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_clean_scrub_stamp":"2024-09-11T08:06:40.591817+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:03:12.280109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.359059+0000","last_change":"2024-09-11T08:06:39.608804+0000","last_active":"2024-09-11T08:06:53.359059+0000","last_peered":"2024-09-11T08:06:53.359059+0000","last_clean":"2024-09-11T08:06:53.359059+0000","last_became_active":"2024-09-11T08:06:39.607177+0000","last_became_peered":"2024-09-11T08:06:39.607177+0000","last_unstale":"2024-09-11T08:06:53.359059+0000","last_undegraded":"2024-09-11T08:06:53.359059+0000","last_fullsized":"2024-09-11T08:06:53.359059+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_clean_scrub_stamp":"2024-09-11T08:06:38.581279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:55:25.120756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.357344+0000","last_change":"2024-09-11T08:06:43.627101+0000","last_active":"2024-09-11T08:06:53.357344+0000","last_peered":"2024-09-11T08:06:53.357344+0000","last_clean":"2024-09-11T08:06:53.357344+0000","last_became_active":"2024-09-11T08:06:43.626809+0000","last_became_peered":"2024-09-11T08:06:43.626809+0000","last_unstale":"2024-09-11T08:06:53.357344+0000","last_undegraded":"2024-09-11T08:06:53.357344+0000","last_fullsized":"2024-09-11T08:06:53.357344+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_clean_scrub_stamp":"2024-09-11T08:06:42.602559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:48:21.116393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"59'1","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.921967+0000","last_change":"2024-09-11T08:06:45.632561+0000","last_active":"2024-09-11T08:06:53.921967+0000","last_peered":"2024-09-11T08:06:53.921967+0000","last_clean":"2024-09-11T08:06:53.921967+0000","last_became_active":"2024-09-11T08:06:45.632248+0000","last_became_peered":"2024-09-11T08:06:45.632248+0000","last_unstale":"2024-09-11T08:06:53.921967+0000","last_undegraded":"2024-09-11T08:06:53.921967+0000","last_fullsized":"2024-09-11T08:06:53.921967+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_clean_scrub_stamp":"2024-09-11T08:06:44.606510+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:36:11.933729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"59'9","reported_seq":32,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.357800+0000","last_change":"2024-09-11T08:06:41.620163+0000","last_active":"2024-09-11T08:06:53.357800+0000","last_peered":"2024-09-11T08:06:53.357800+0000","last_clean":"2024-09-11T08:06:53.357800+0000","last_became_active":"2024-09-11T08:06:41.619582+0000","last_became_peered":"2024-09-11T08:06:41.619582+0000","last_unstale":"2024-09-11T08:06:53.357800+0000","last_undegraded":"2024-09-11T08:06:53.357800+0000","last_fullsized":"2024-09-11T08:06:53.357800+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_clean_scrub_stamp":"2024-09-11T08:06:40.591817+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:46:55.251605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.923560+0000","last_change":"2024-09-11T08:06:39.606228+0000","last_active":"2024-09-11T08:06:53.923560+0000","last_peered":"2024-09-11T08:06:53.923560+0000","last_clean":"2024-09-11T08:06:53.923560+0000","last_became_active":"2024-09-11T08:06:39.605978+0000","last_became_peered":"2024-09-11T08:06:39.605978+0000","last_unstale":"2024-09-11T08:06:53.923560+0000","last_undegraded":"2024-09-11T08:06:53.923560+0000","last_fullsized":"2024-09-11T08:06:53.923560+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_clean_scrub_stamp":"2024-09-11T08:06:38.581279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:20:04.792419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.357122+0000","last_change":"2024-09-11T08:06:43.621932+0000","last_active":"2024-09-11T08:06:53.357122+0000","last_peered":"2024-09-11T08:06:53.357122+0000","last_clean":"2024-09-11T08:06:53.357122+0000","last_became_active":"2024-09-11T08:06:43.621631+0000","last_became_peered":"2024-09-11T08:06:43.621631+0000","last_unstale":"2024-09-11T08:06:53.357122+0000","last_undegraded":"2024-09-11T08:06:53.357122+0000","last_fullsized":"2024-09-11T08:06:53.357122+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_clean_scrub_stamp":"2024-09-11T08:06:42.602559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:00:18.710953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.358253+0000","last_change":"2024-09-11T08:06:45.629092+0000","last_active":"2024-09-11T08:06:53.358253+0000","last_peered":"2024-09-11T08:06:53.358253+0000","last_clean":"2024-09-11T08:06:53.358253+0000","last_became_active":"2024-09-11T08:06:45.628810+0000","last_became_peered":"2024-09-11T08:06:45.628810+0000","last_unstale":"2024-09-11T08:06:53.358253+0000","last_undegraded":"2024-09-11T08:06:53.358253+0000","last_fullsized":"2024-09-11T08:06:53.358253+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_clean_scrub_stamp":"2024-09-11T08:06:44.606510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:31:17.639950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"59'15","reported_seq":41,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.923870+0000","last_change":"2024-09-11T08:06:42.465052+0000","last_active":"2024-09-11T08:06:53.923870+0000","last_peered":"2024-09-11T08:06:53.923870+0000","last_clean":"2024-09-11T08:06:53.923870+0000","last_became_active":"2024-09-11T08:06:42.464746+0000","last_became_peered":"2024-09-11T08:06:42.464746+0000","last_unstale":"2024-09-11T08:06:53.923870+0000","last_undegraded":"2024-09-11T08:06:53.923870+0000","last_fullsized":"2024-09-11T08:06:53.923870+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_clean_scrub_stamp":"2024-09-11T08:06:40.591817+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:10:24.337047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.358070+0000","last_change":"2024-09-11T08:06:39.602562+0000","last_active":"2024-09-11T08:06:53.358070+0000","last_peered":"2024-09-11T08:06:53.358070+0000","last_clean":"2024-09-11T08:06:53.358070+0000","last_became_active":"2024-09-11T08:06:39.602202+0000","last_became_peered":"2024-09-11T08:06:39.602202+0000","last_unstale":"2024-09-11T08:06:53.358070+0000","last_undegraded":"2024-09-11T08:06:53.358070+0000","last_fullsized":"2024-09-11T08:06:53.358070+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_clean_scrub_stamp":"2024-09-11T08:06:38.581279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:05:15.086841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"59'8","reported_seq":38,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:07:53.123133+0000","last_change":"2024-09-11T08:06:43.617812+0000","last_active":"2024-09-11T08:07:53.123133+0000","last_peered":"2024-09-11T08:07:53.123133+0000","last_clean":"2024-09-11T08:07:53.123133+0000","last_became_active":"2024-09-11T08:06:43.617529+0000","last_became_peered":"2024-09-11T08:06:43.617529+0000","last_unstale":"2024-09-11T08:07:53.123133+0000","last_undegraded":"2024-09-11T08:07:53.123133+0000","last_fullsized":"2024-09-11T08:07:53.123133+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_clean_scrub_stamp":"2024-09-11T08:06:42.602559+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:17:25.433041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.923190+0000","last_change":"2024-09-11T08:06:46.174033+0000","last_active":"2024-09-11T08:06:53.923190+0000","last_peered":"2024-09-11T08:06:53.923190+0000","last_clean":"2024-09-11T08:06:53.923190+0000","last_became_active":"2024-09-11T08:06:46.173667+0000","last_became_peered":"2024-09-11T08:06:46.173667+0000","last_unstale":"2024-09-11T08:06:53.923190+0000","last_undegraded":"2024-09-11T08:06:53.923190+0000","last_fullsized":"2024-09-11T08:06:53.923190+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_clean_scrub_stamp":"2024-09-11T08:06:44.606510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:06:17.689074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"59'12","reported_seq":39,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.355965+0000","last_change":"2024-09-11T08:06:41.620218+0000","last_active":"2024-09-11T08:06:53.355965+0000","last_peered":"2024-09-11T08:06:53.355965+0000","last_clean":"2024-09-11T08:06:53.355965+0000","last_became_active":"2024-09-11T08:06:41.619544+0000","last_became_peered":"2024-09-11T08:06:41.619544+0000","last_unstale":"2024-09-11T08:06:53.355965+0000","last_undegraded":"2024-09-11T08:06:53.355965+0000","last_fullsized":"2024-09-11T08:06:53.355965+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_clean_scrub_stamp":"2024-09-11T08:06:40.591817+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:49:09.046422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"52'3","reported_seq":44,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.356573+0000","last_change":"2024-09-11T08:06:39.607264+0000","last_active":"2024-09-11T08:06:53.356573+0000","last_peered":"2024-09-11T08:06:53.356573+0000","last_clean":"2024-09-11T08:06:53.356573+0000","last_became_active":"2024-09-11T08:06:39.606994+0000","last_became_peered":"2024-09-11T08:06:39.606994+0000","last_unstale":"2024-09-11T08:06:53.356573+0000","last_undegraded":"2024-09-11T08:06:53.356573+0000","last_fullsized":"2024-09-11T08:06:53.356573+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_clean_scrub_stamp":"2024-09-11T08:06:38.581279+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:29:01.420487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":23,"num_read_kb":23,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.356887+0000","last_change":"2024-09-11T08:06:43.622700+0000","last_active":"2024-09-11T08:06:53.356887+0000","last_peered":"2024-09-11T08:06:53.356887+0000","last_clean":"2024-09-11T08:06:53.356887+0000","last_became_active":"2024-09-11T08:06:43.622338+0000","last_became_peered":"2024-09-11T08:06:43.622338+0000","last_unstale":"2024-09-11T08:06:53.356887+0000","last_undegraded":"2024-09-11T08:06:53.356887+0000","last_fullsized":"2024-09-11T08:06:53.356887+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_clean_scrub_stamp":"2024-09-11T08:06:42.602559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:27:44.475248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.504909+0000","last_change":"2024-09-11T08:06:45.627104+0000","last_active":"2024-09-11T08:06:53.504909+0000","last_peered":"2024-09-11T08:06:53.504909+0000","last_clean":"2024-09-11T08:06:53.504909+0000","last_became_active":"2024-09-11T08:06:45.626776+0000","last_became_peered":"2024-09-11T08:06:45.626776+0000","last_unstale":"2024-09-11T08:06:53.504909+0000","last_undegraded":"2024-09-11T08:06:53.504909+0000","last_fullsized":"2024-09-11T08:06:53.504909+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_clean_scrub_stamp":"2024-09-11T08:06:44.606510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:27:09.664925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"59'9","reported_seq":32,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.358500+0000","last_change":"2024-09-11T08:06:42.464514+0000","last_active":"2024-09-11T08:06:53.358500+0000","last_peered":"2024-09-11T08:06:53.358500+0000","last_clean":"2024-09-11T08:06:53.358500+0000","last_became_active":"2024-09-11T08:06:42.463984+0000","last_became_peered":"2024-09-11T08:06:42.463984+0000","last_unstale":"2024-09-11T08:06:53.358500+0000","last_undegraded":"2024-09-11T08:06:53.358500+0000","last_fullsized":"2024-09-11T08:06:53.358500+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_clean_scrub_stamp":"2024-09-11T08:06:40.591817+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:22:31.967107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.356963+0000","last_change":"2024-09-11T08:06:39.604875+0000","last_active":"2024-09-11T08:06:53.356963+0000","last_peered":"2024-09-11T08:06:53.356963+0000","last_clean":"2024-09-11T08:06:53.356963+0000","last_became_active":"2024-09-11T08:06:39.604485+0000","last_became_peered":"2024-09-11T08:06:39.604485+0000","last_unstale":"2024-09-11T08:06:53.356963+0000","last_undegraded":"2024-09-11T08:06:53.356963+0000","last_fullsized":"2024-09-11T08:06:53.356963+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_clean_scrub_stamp":"2024-09-11T08:06:38.581279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:32:49.491454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.505524+0000","last_change":"2024-09-11T08:06:43.621732+0000","last_active":"2024-09-11T08:06:53.505524+0000","last_peered":"2024-09-11T08:06:53.505524+0000","last_clean":"2024-09-11T08:06:53.505524+0000","last_became_active":"2024-09-11T08:06:43.621463+0000","last_became_peered":"2024-09-11T08:06:43.621463+0000","last_unstale":"2024-09-11T08:06:53.505524+0000","last_undegraded":"2024-09-11T08:06:53.505524+0000","last_fullsized":"2024-09-11T08:06:53.505524+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_clean_scrub_stamp":"2024-09-11T08:06:42.602559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:03:39.556106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.358612+0000","last_change":"2024-09-11T08:06:45.628869+0000","last_active":"2024-09-11T08:06:53.358612+0000","last_peered":"2024-09-11T08:06:53.358612+0000","last_clean":"2024-09-11T08:06:53.358612+0000","last_became_active":"2024-09-11T08:06:45.628457+0000","last_became_peered":"2024-09-11T08:06:45.628457+0000","last_unstale":"2024-09-11T08:06:53.358612+0000","last_undegraded":"2024-09-11T08:06:53.358612+0000","last_fullsized":"2024-09-11T08:06:53.358612+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_clean_scrub_stamp":"2024-09-11T08:06:44.606510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:30:45.531791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.922153+0000","last_change":"2024-09-11T08:06:39.606516+0000","last_active":"2024-09-11T08:06:53.922153+0000","last_peered":"2024-09-11T08:06:53.922153+0000","last_clean":"2024-09-11T08:06:53.922153+0000","last_became_active":"2024-09-11T08:06:39.606173+0000","last_became_peered":"2024-09-11T08:06:39.606173+0000","last_unstale":"2024-09-11T08:06:53.922153+0000","last_undegraded":"2024-09-11T08:06:53.922153+0000","last_fullsized":"2024-09-11T08:06:53.922153+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_clean_scrub_stamp":"2024-09-11T08:06:38.581279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:42:30.212709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"59'6","reported_seq":25,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.505593+0000","last_change":"2024-09-11T08:06:42.463567+0000","last_active":"2024-09-11T08:06:53.505593+0000","last_peered":"2024-09-11T08:06:53.505593+0000","last_clean":"2024-09-11T08:06:53.505593+0000","last_became_active":"2024-09-11T08:06:42.463290+0000","last_became_peered":"2024-09-11T08:06:42.463290+0000","last_unstale":"2024-09-11T08:06:53.505593+0000","last_undegraded":"2024-09-11T08:06:53.505593+0000","last_fullsized":"2024-09-11T08:06:53.505593+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_clean_scrub_stamp":"2024-09-11T08:06:40.591817+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:34:30.894253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.923719+0000","last_change":"2024-09-11T08:06:43.627168+0000","last_active":"2024-09-11T08:06:53.923719+0000","last_peered":"2024-09-11T08:06:53.923719+0000","last_clean":"2024-09-11T08:06:53.923719+0000","last_became_active":"2024-09-11T08:06:43.626710+0000","last_became_peered":"2024-09-11T08:06:43.626710+0000","last_unstale":"2024-09-11T08:06:53.923719+0000","last_undegraded":"2024-09-11T08:06:53.923719+0000","last_fullsized":"2024-09-11T08:06:53.923719+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_clean_scrub_stamp":"2024-09-11T08:06:42.602559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:41:53.559126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.922464+0000","last_change":"2024-09-11T08:06:46.175979+0000","last_active":"2024-09-11T08:06:53.922464+0000","last_peered":"2024-09-11T08:06:53.922464+0000","last_clean":"2024-09-11T08:06:53.922464+0000","last_became_active":"2024-09-11T08:06:46.175692+0000","last_became_peered":"2024-09-11T08:06:46.175692+0000","last_unstale":"2024-09-11T08:06:53.922464+0000","last_undegraded":"2024-09-11T08:06:53.922464+0000","last_fullsized":"2024-09-11T08:06:53.922464+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_clean_scrub_stamp":"2024-09-11T08:06:44.606510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:54:55.090852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"59'10","reported_seq":31,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.505563+0000","last_change":"2024-09-11T08:06:42.463299+0000","last_active":"2024-09-11T08:06:53.505563+0000","last_peered":"2024-09-11T08:06:53.505563+0000","last_clean":"2024-09-11T08:06:53.505563+0000","last_became_active":"2024-09-11T08:06:42.462909+0000","last_became_peered":"2024-09-11T08:06:42.462909+0000","last_unstale":"2024-09-11T08:06:53.505563+0000","last_undegraded":"2024-09-11T08:06:53.505563+0000","last_fullsized":"2024-09-11T08:06:53.505563+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_clean_scrub_stamp":"2024-09-11T08:06:40.591817+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:52:15.756396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.923475+0000","last_change":"2024-09-11T08:06:39.605030+0000","last_active":"2024-09-11T08:06:53.923475+0000","last_peered":"2024-09-11T08:06:53.923475+0000","last_clean":"2024-09-11T08:06:53.923475+0000","last_became_active":"2024-09-11T08:06:39.604758+0000","last_became_peered":"2024-09-11T08:06:39.604758+0000","last_unstale":"2024-09-11T08:06:53.923475+0000","last_undegraded":"2024-09-11T08:06:53.923475+0000","last_fullsized":"2024-09-11T08:06:53.923475+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_clean_scrub_stamp":"2024-09-11T08:06:38.581279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:21:23.445399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"59'8","reported_seq":38,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:07:53.123135+0000","last_change":"2024-09-11T08:06:43.626946+0000","last_active":"2024-09-11T08:07:53.123135+0000","last_peered":"2024-09-11T08:07:53.123135+0000","last_clean":"2024-09-11T08:07:53.123135+0000","last_became_active":"2024-09-11T08:06:43.626013+0000","last_became_peered":"2024-09-11T08:06:43.626013+0000","last_unstale":"2024-09-11T08:07:53.123135+0000","last_undegraded":"2024-09-11T08:07:53.123135+0000","last_fullsized":"2024-09-11T08:07:53.123135+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_clean_scrub_stamp":"2024-09-11T08:06:42.602559+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:24:37.994554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.358182+0000","last_change":"2024-09-11T08:06:45.628724+0000","last_active":"2024-09-11T08:06:53.358182+0000","last_peered":"2024-09-11T08:06:53.358182+0000","last_clean":"2024-09-11T08:06:53.358182+0000","last_became_active":"2024-09-11T08:06:45.627922+0000","last_became_peered":"2024-09-11T08:06:45.627922+0000","last_unstale":"2024-09-11T08:06:53.358182+0000","last_undegraded":"2024-09-11T08:06:53.358182+0000","last_fullsized":"2024-09-11T08:06:53.358182+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_clean_scrub_stamp":"2024-09-11T08:06:44.606510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:51:02.980182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"59'9","reported_seq":32,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.923973+0000","last_change":"2024-09-11T08:06:42.464248+0000","last_active":"2024-09-11T08:06:53.923973+0000","last_peered":"2024-09-11T08:06:53.923973+0000","last_clean":"2024-09-11T08:06:53.923973+0000","last_became_active":"2024-09-11T08:06:42.463958+0000","last_became_peered":"2024-09-11T08:06:42.463958+0000","last_unstale":"2024-09-11T08:06:53.923973+0000","last_undegraded":"2024-09-11T08:06:53.923973+0000","last_fullsized":"2024-09-11T08:06:53.923973+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_clean_scrub_stamp":"2024-09-11T08:06:40.591817+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:38:51.336334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.358038+0000","last_change":"2024-09-11T08:06:39.607967+0000","last_active":"2024-09-11T08:06:53.358038+0000","last_peered":"2024-09-11T08:06:53.358038+0000","last_clean":"2024-09-11T08:06:53.358038+0000","last_became_active":"2024-09-11T08:06:39.607693+0000","last_became_peered":"2024-09-11T08:06:39.607693+0000","last_unstale":"2024-09-11T08:06:53.358038+0000","last_undegraded":"2024-09-11T08:06:53.358038+0000","last_fullsized":"2024-09-11T08:06:53.358038+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_clean_scrub_stamp":"2024-09-11T08:06:38.581279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:11.296892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"59'8","reported_seq":38,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:07:53.122896+0000","last_change":"2024-09-11T08:06:43.620860+0000","last_active":"2024-09-11T08:07:53.122896+0000","last_peered":"2024-09-11T08:07:53.122896+0000","last_clean":"2024-09-11T08:07:53.122896+0000","last_became_active":"2024-09-11T08:06:43.620572+0000","last_became_peered":"2024-09-11T08:06:43.620572+0000","last_unstale":"2024-09-11T08:07:53.122896+0000","last_undegraded":"2024-09-11T08:07:53.122896+0000","last_fullsized":"2024-09-11T08:07:53.122896+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_clean_scrub_stamp":"2024-09-11T08:06:42.602559+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:03:09.496213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.356486+0000","last_change":"2024-09-11T08:06:45.625629+0000","last_active":"2024-09-11T08:06:53.356486+0000","last_peered":"2024-09-11T08:06:53.356486+0000","last_clean":"2024-09-11T08:06:53.356486+0000","last_became_active":"2024-09-11T08:06:45.625334+0000","last_became_peered":"2024-09-11T08:06:45.625334+0000","last_unstale":"2024-09-11T08:06:53.356486+0000","last_undegraded":"2024-09-11T08:06:53.356486+0000","last_fullsized":"2024-09-11T08:06:53.356486+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_clean_scrub_stamp":"2024-09-11T08:06:44.606510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:52:42.198104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"59'9","reported_seq":32,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.922990+0000","last_change":"2024-09-11T08:06:41.618370+0000","last_active":"2024-09-11T08:06:53.922990+0000","last_peered":"2024-09-11T08:06:53.922990+0000","last_clean":"2024-09-11T08:06:53.922990+0000","last_became_active":"2024-09-11T08:06:41.616919+0000","last_became_peered":"2024-09-11T08:06:41.616919+0000","last_unstale":"2024-09-11T08:06:53.922990+0000","last_undegraded":"2024-09-11T08:06:53.922990+0000","last_fullsized":"2024-09-11T08:06:53.922990+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_clean_scrub_stamp":"2024-09-11T08:06:40.591817+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:44:01.768714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.922658+0000","last_change":"2024-09-11T08:06:39.601409+0000","last_active":"2024-09-11T08:06:53.922658+0000","last_peered":"2024-09-11T08:06:53.922658+0000","last_clean":"2024-09-11T08:06:53.922658+0000","last_became_active":"2024-09-11T08:06:39.600679+0000","last_became_peered":"2024-09-11T08:06:39.600679+0000","last_unstale":"2024-09-11T08:06:53.922658+0000","last_undegraded":"2024-09-11T08:06:53.922658+0000","last_fullsized":"2024-09-11T08:06:53.922658+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_clean_scrub_stamp":"2024-09-11T08:06:38.581279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:27:17.806820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.504985+0000","last_change":"2024-09-11T08:06:43.624960+0000","last_active":"2024-09-11T08:06:53.504985+0000","last_peered":"2024-09-11T08:06:53.504985+0000","last_clean":"2024-09-11T08:06:53.504985+0000","last_became_active":"2024-09-11T08:06:43.624654+0000","last_became_peered":"2024-09-11T08:06:43.624654+0000","last_unstale":"2024-09-11T08:06:53.504985+0000","last_undegraded":"2024-09-11T08:06:53.504985+0000","last_fullsized":"2024-09-11T08:06:53.504985+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_clean_scrub_stamp":"2024-09-11T08:06:42.602559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:35:56.009761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.356843+0000","last_change":"2024-09-11T08:06:45.630924+0000","last_active":"2024-09-11T08:06:53.356843+0000","last_peered":"2024-09-11T08:06:53.356843+0000","last_clean":"2024-09-11T08:06:53.356843+0000","last_became_active":"2024-09-11T08:06:45.630564+0000","last_became_peered":"2024-09-11T08:06:45.630564+0000","last_unstale":"2024-09-11T08:06:53.356843+0000","last_undegraded":"2024-09-11T08:06:53.356843+0000","last_fullsized":"2024-09-11T08:06:53.356843+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_clean_scrub_stamp":"2024-09-11T08:06:44.606510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:22:33.148769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"59'11","reported_seq":35,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.355982+0000","last_change":"2024-09-11T08:06:41.620509+0000","last_active":"2024-09-11T08:06:53.355982+0000","last_peered":"2024-09-11T08:06:53.355982+0000","last_clean":"2024-09-11T08:06:53.355982+0000","last_became_active":"2024-09-11T08:06:41.620024+0000","last_became_peered":"2024-09-11T08:06:41.620024+0000","last_unstale":"2024-09-11T08:06:53.355982+0000","last_undegraded":"2024-09-11T08:06:53.355982+0000","last_fullsized":"2024-09-11T08:06:53.355982+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_clean_scrub_stamp":"2024-09-11T08:06:40.591817+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:20.707096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.923522+0000","last_change":"2024-09-11T08:06:39.606984+0000","last_active":"2024-09-11T08:06:53.923522+0000","last_peered":"2024-09-11T08:06:53.923522+0000","last_clean":"2024-09-11T08:06:53.923522+0000","last_became_active":"2024-09-11T08:06:39.606710+0000","last_became_peered":"2024-09-11T08:06:39.606710+0000","last_unstale":"2024-09-11T08:06:53.923522+0000","last_undegraded":"2024-09-11T08:06:53.923522+0000","last_fullsized":"2024-09-11T08:06:53.923522+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_clean_scrub_stamp":"2024-09-11T08:06:38.581279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:45:54.406289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.923060+0000","last_change":"2024-09-11T08:06:43.629521+0000","last_active":"2024-09-11T08:06:53.923060+0000","last_peered":"2024-09-11T08:06:53.923060+0000","last_clean":"2024-09-11T08:06:53.923060+0000","last_became_active":"2024-09-11T08:06:43.629214+0000","last_became_peered":"2024-09-11T08:06:43.629214+0000","last_unstale":"2024-09-11T08:06:53.923060+0000","last_undegraded":"2024-09-11T08:06:53.923060+0000","last_fullsized":"2024-09-11T08:06:53.923060+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_clean_scrub_stamp":"2024-09-11T08:06:42.602559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:38:34.846754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.504636+0000","last_change":"2024-09-11T08:06:45.630229+0000","last_active":"2024-09-11T08:06:53.504636+0000","last_peered":"2024-09-11T08:06:53.504636+0000","last_clean":"2024-09-11T08:06:53.504636+0000","last_became_active":"2024-09-11T08:06:45.629929+0000","last_became_peered":"2024-09-11T08:06:45.629929+0000","last_unstale":"2024-09-11T08:06:53.504636+0000","last_undegraded":"2024-09-11T08:06:53.504636+0000","last_fullsized":"2024-09-11T08:06:53.504636+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_clean_scrub_stamp":"2024-09-11T08:06:44.606510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:18:12.022222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"59'4","reported_seq":22,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.504677+0000","last_change":"2024-09-11T08:06:41.621102+0000","last_active":"2024-09-11T08:06:53.504677+0000","last_peered":"2024-09-11T08:06:53.504677+0000","last_clean":"2024-09-11T08:06:53.504677+0000","last_became_active":"2024-09-11T08:06:41.620784+0000","last_became_peered":"2024-09-11T08:06:41.620784+0000","last_unstale":"2024-09-11T08:06:53.504677+0000","last_undegraded":"2024-09-11T08:06:53.504677+0000","last_fullsized":"2024-09-11T08:06:53.504677+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_clean_scrub_stamp":"2024-09-11T08:06:40.591817+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:59:40.715245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.923844+0000","last_change":"2024-09-11T08:06:39.605989+0000","last_active":"2024-09-11T08:06:53.923844+0000","last_peered":"2024-09-11T08:06:53.923844+0000","last_clean":"2024-09-11T08:06:53.923844+0000","last_became_active":"2024-09-11T08:06:39.605610+0000","last_became_peered":"2024-09-11T08:06:39.605610+0000","last_unstale":"2024-09-11T08:06:53.923844+0000","last_undegraded":"2024-09-11T08:06:53.923844+0000","last_fullsized":"2024-09-11T08:06:53.923844+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_clean_scrub_stamp":"2024-09-11T08:06:38.581279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:29:10.795022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.921898+0000","last_change":"2024-09-11T08:06:43.621216+0000","last_active":"2024-09-11T08:06:53.921898+0000","last_peered":"2024-09-11T08:06:53.921898+0000","last_clean":"2024-09-11T08:06:53.921898+0000","last_became_active":"2024-09-11T08:06:43.620960+0000","last_became_peered":"2024-09-11T08:06:43.620960+0000","last_unstale":"2024-09-11T08:06:53.921898+0000","last_undegraded":"2024-09-11T08:06:53.921898+0000","last_fullsized":"2024-09-11T08:06:53.921898+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_clean_scrub_stamp":"2024-09-11T08:06:42.602559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:10:15.343370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.921911+0000","last_change":"2024-09-11T08:06:45.628406+0000","last_active":"2024-09-11T08:06:53.921911+0000","last_peered":"2024-09-11T08:06:53.921911+0000","last_clean":"2024-09-11T08:06:53.921911+0000","last_became_active":"2024-09-11T08:06:45.627358+0000","last_became_peered":"2024-09-11T08:06:45.627358+0000","last_unstale":"2024-09-11T08:06:53.921911+0000","last_undegraded":"2024-09-11T08:06:53.921911+0000","last_fullsized":"2024-09-11T08:06:53.921911+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_clean_scrub_stamp":"2024-09-11T08:06:44.606510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:12:15.432613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"59'11","reported_seq":35,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.504912+0000","last_change":"2024-09-11T08:06:42.464194+0000","last_active":"2024-09-11T08:06:53.504912+0000","last_peered":"2024-09-11T08:06:53.504912+0000","last_clean":"2024-09-11T08:06:53.504912+0000","last_became_active":"2024-09-11T08:06:42.463040+0000","last_became_peered":"2024-09-11T08:06:42.463040+0000","last_unstale":"2024-09-11T08:06:53.504912+0000","last_undegraded":"2024-09-11T08:06:53.504912+0000","last_fullsized":"2024-09-11T08:06:53.504912+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_clean_scrub_stamp":"2024-09-11T08:06:40.591817+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:20:31.230643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.922924+0000","last_change":"2024-09-11T08:06:39.603395+0000","last_active":"2024-09-11T08:06:53.922924+0000","last_peered":"2024-09-11T08:06:53.922924+0000","last_clean":"2024-09-11T08:06:53.922924+0000","last_became_active":"2024-09-11T08:06:39.602940+0000","last_became_peered":"2024-09-11T08:06:39.602940+0000","last_unstale":"2024-09-11T08:06:53.922924+0000","last_undegraded":"2024-09-11T08:06:53.922924+0000","last_fullsized":"2024-09-11T08:06:53.922924+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_clean_scrub_stamp":"2024-09-11T08:06:38.581279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:20:13.814946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.922672+0000","last_change":"2024-09-11T08:06:43.619135+0000","last_active":"2024-09-11T08:06:53.922672+0000","last_peered":"2024-09-11T08:06:53.922672+0000","last_clean":"2024-09-11T08:06:53.922672+0000","last_became_active":"2024-09-11T08:06:43.618865+0000","last_became_peered":"2024-09-11T08:06:43.618865+0000","last_unstale":"2024-09-11T08:06:53.922672+0000","last_undegraded":"2024-09-11T08:06:53.922672+0000","last_fullsized":"2024-09-11T08:06:53.922672+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_clean_scrub_stamp":"2024-09-11T08:06:42.602559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:51:03.011293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.504743+0000","last_change":"2024-09-11T08:06:46.174680+0000","last_active":"2024-09-11T08:06:53.504743+0000","last_peered":"2024-09-11T08:06:53.504743+0000","last_clean":"2024-09-11T08:06:53.504743+0000","last_became_active":"2024-09-11T08:06:46.174359+0000","last_became_peered":"2024-09-11T08:06:46.174359+0000","last_unstale":"2024-09-11T08:06:53.504743+0000","last_undegraded":"2024-09-11T08:06:53.504743+0000","last_fullsized":"2024-09-11T08:06:53.504743+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_clean_scrub_stamp":"2024-09-11T08:06:44.606510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:19:21.822289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"59'1","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.921790+0000","last_change":"2024-09-11T08:06:45.628156+0000","last_active":"2024-09-11T08:06:53.921790+0000","last_peered":"2024-09-11T08:06:53.921790+0000","last_clean":"2024-09-11T08:06:53.921790+0000","last_became_active":"2024-09-11T08:06:45.627877+0000","last_became_peered":"2024-09-11T08:06:45.627877+0000","last_unstale":"2024-09-11T08:06:53.921790+0000","last_undegraded":"2024-09-11T08:06:53.921790+0000","last_fullsized":"2024-09-11T08:06:53.921790+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_clean_scrub_stamp":"2024-09-11T08:06:44.606510+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:28:49.200951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.923962+0000","last_change":"2024-09-11T08:06:39.604005+0000","last_active":"2024-09-11T08:06:53.923962+0000","last_peered":"2024-09-11T08:06:53.923962+0000","last_clean":"2024-09-11T08:06:53.923962+0000","last_became_active":"2024-09-11T08:06:39.603733+0000","last_became_peered":"2024-09-11T08:06:39.603733+0000","last_unstale":"2024-09-11T08:06:53.923962+0000","last_undegraded":"2024-09-11T08:06:53.923962+0000","last_fullsized":"2024-09-11T08:06:53.923962+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_clean_scrub_stamp":"2024-09-11T08:06:38.581279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:01.485991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"59'10","reported_seq":31,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.358436+0000","last_change":"2024-09-11T08:06:42.464747+0000","last_active":"2024-09-11T08:06:53.358436+0000","last_peered":"2024-09-11T08:06:53.358436+0000","last_clean":"2024-09-11T08:06:53.358436+0000","last_became_active":"2024-09-11T08:06:42.464432+0000","last_became_peered":"2024-09-11T08:06:42.464432+0000","last_unstale":"2024-09-11T08:06:53.358436+0000","last_undegraded":"2024-09-11T08:06:53.358436+0000","last_fullsized":"2024-09-11T08:06:53.358436+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_clean_scrub_stamp":"2024-09-11T08:06:40.591817+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:46:26.124072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"59'8","reported_seq":41,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:07:53.123124+0000","last_change":"2024-09-11T08:06:43.619638+0000","last_active":"2024-09-11T08:07:53.123124+0000","last_peered":"2024-09-11T08:07:53.123124+0000","last_clean":"2024-09-11T08:07:53.123124+0000","last_became_active":"2024-09-11T08:06:43.619337+0000","last_became_peered":"2024-09-11T08:06:43.619337+0000","last_unstale":"2024-09-11T08:07:53.123124+0000","last_undegraded":"2024-09-11T08:07:53.123124+0000","last_fullsized":"2024-09-11T08:07:53.123124+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_clean_scrub_stamp":"2024-09-11T08:06:42.602559+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:55:38.708691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.922291+0000","last_change":"2024-09-11T08:06:45.627949+0000","last_active":"2024-09-11T08:06:53.922291+0000","last_peered":"2024-09-11T08:06:53.922291+0000","last_clean":"2024-09-11T08:06:53.922291+0000","last_became_active":"2024-09-11T08:06:45.627633+0000","last_became_peered":"2024-09-11T08:06:45.627633+0000","last_unstale":"2024-09-11T08:06:53.922291+0000","last_undegraded":"2024-09-11T08:06:53.922291+0000","last_fullsized":"2024-09-11T08:06:53.922291+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_clean_scrub_stamp":"2024-09-11T08:06:44.606510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:34:40.595042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.505251+0000","last_change":"2024-09-11T08:06:39.605550+0000","last_active":"2024-09-11T08:06:53.505251+0000","last_peered":"2024-09-11T08:06:53.505251+0000","last_clean":"2024-09-11T08:06:53.505251+0000","last_became_active":"2024-09-11T08:06:39.605246+0000","last_became_peered":"2024-09-11T08:06:39.605246+0000","last_unstale":"2024-09-11T08:06:53.505251+0000","last_undegraded":"2024-09-11T08:06:53.505251+0000","last_fullsized":"2024-09-11T08:06:53.505251+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_clean_scrub_stamp":"2024-09-11T08:06:38.581279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:02:04.921114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"59'11","reported_seq":35,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.922854+0000","last_change":"2024-09-11T08:06:41.620114+0000","last_active":"2024-09-11T08:06:53.922854+0000","last_peered":"2024-09-11T08:06:53.922854+0000","last_clean":"2024-09-11T08:06:53.922854+0000","last_became_active":"2024-09-11T08:06:41.619861+0000","last_became_peered":"2024-09-11T08:06:41.619861+0000","last_unstale":"2024-09-11T08:06:53.922854+0000","last_undegraded":"2024-09-11T08:06:53.922854+0000","last_fullsized":"2024-09-11T08:06:53.922854+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_clean_scrub_stamp":"2024-09-11T08:06:40.591817+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:50:47.601811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.358352+0000","last_change":"2024-09-11T08:06:43.614213+0000","last_active":"2024-09-11T08:06:53.358352+0000","last_peered":"2024-09-11T08:06:53.358352+0000","last_clean":"2024-09-11T08:06:53.358352+0000","last_became_active":"2024-09-11T08:06:43.613927+0000","last_became_peered":"2024-09-11T08:06:43.613927+0000","last_unstale":"2024-09-11T08:06:53.358352+0000","last_undegraded":"2024-09-11T08:06:53.358352+0000","last_fullsized":"2024-09-11T08:06:53.358352+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_clean_scrub_stamp":"2024-09-11T08:06:42.602559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:15:07.385448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.356268+0000","last_change":"2024-09-11T08:06:46.172587+0000","last_active":"2024-09-11T08:06:53.356268+0000","last_peered":"2024-09-11T08:06:53.356268+0000","last_clean":"2024-09-11T08:06:53.356268+0000","last_became_active":"2024-09-11T08:06:46.172235+0000","last_became_peered":"2024-09-11T08:06:46.172235+0000","last_unstale":"2024-09-11T08:06:53.356268+0000","last_undegraded":"2024-09-11T08:06:53.356268+0000","last_fullsized":"2024-09-11T08:06:53.356268+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_clean_scrub_stamp":"2024-09-11T08:06:44.606510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:13:30.787527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.922892+0000","last_change":"2024-09-11T08:06:39.600828+0000","last_active":"2024-09-11T08:06:53.922892+0000","last_peered":"2024-09-11T08:06:53.922892+0000","last_clean":"2024-09-11T08:06:53.922892+0000","last_became_active":"2024-09-11T08:06:39.600572+0000","last_became_peered":"2024-09-11T08:06:39.600572+0000","last_unstale":"2024-09-11T08:06:53.922892+0000","last_undegraded":"2024-09-11T08:06:53.922892+0000","last_fullsized":"2024-09-11T08:06:53.922892+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_clean_scrub_stamp":"2024-09-11T08:06:38.581279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:39:20.392722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"59'15","reported_seq":41,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.358968+0000","last_change":"2024-09-11T08:06:41.618530+0000","last_active":"2024-09-11T08:06:53.358968+0000","last_peered":"2024-09-11T08:06:53.358968+0000","last_clean":"2024-09-11T08:06:53.358968+0000","last_became_active":"2024-09-11T08:06:41.618210+0000","last_became_peered":"2024-09-11T08:06:41.618210+0000","last_unstale":"2024-09-11T08:06:53.358968+0000","last_undegraded":"2024-09-11T08:06:53.358968+0000","last_fullsized":"2024-09-11T08:06:53.358968+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_clean_scrub_stamp":"2024-09-11T08:06:40.591817+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:21:39.506662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.923351+0000","last_change":"2024-09-11T08:06:43.623753+0000","last_active":"2024-09-11T08:06:53.923351+0000","last_peered":"2024-09-11T08:06:53.923351+0000","last_clean":"2024-09-11T08:06:53.923351+0000","last_became_active":"2024-09-11T08:06:43.623433+0000","last_became_peered":"2024-09-11T08:06:43.623433+0000","last_unstale":"2024-09-11T08:06:53.923351+0000","last_undegraded":"2024-09-11T08:06:53.923351+0000","last_fullsized":"2024-09-11T08:06:53.923351+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_clean_scrub_stamp":"2024-09-11T08:06:42.602559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:39:51.475323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.505402+0000","last_change":"2024-09-11T08:06:46.172978+0000","last_active":"2024-09-11T08:06:53.505402+0000","last_peered":"2024-09-11T08:06:53.505402+0000","last_clean":"2024-09-11T08:06:53.505402+0000","last_became_active":"2024-09-11T08:06:46.172348+0000","last_became_peered":"2024-09-11T08:06:46.172348+0000","last_unstale":"2024-09-11T08:06:53.505402+0000","last_undegraded":"2024-09-11T08:06:53.505402+0000","last_fullsized":"2024-09-11T08:06:53.505402+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:44.606510+0000","last_clean_scrub_stamp":"2024-09-11T08:06:44.606510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:03:05.325184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.505132+0000","last_change":"2024-09-11T08:06:39.602456+0000","last_active":"2024-09-11T08:06:53.505132+0000","last_peered":"2024-09-11T08:06:53.505132+0000","last_clean":"2024-09-11T08:06:53.505132+0000","last_became_active":"2024-09-11T08:06:39.601844+0000","last_became_peered":"2024-09-11T08:06:39.601844+0000","last_unstale":"2024-09-11T08:06:53.505132+0000","last_undegraded":"2024-09-11T08:06:53.505132+0000","last_fullsized":"2024-09-11T08:06:53.505132+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:38.581279+0000","last_clean_scrub_stamp":"2024-09-11T08:06:38.581279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:45:49.618077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"59'12","reported_seq":39,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.505247+0000","last_change":"2024-09-11T08:06:41.620881+0000","last_active":"2024-09-11T08:06:53.505247+0000","last_peered":"2024-09-11T08:06:53.505247+0000","last_clean":"2024-09-11T08:06:53.505247+0000","last_became_active":"2024-09-11T08:06:41.620564+0000","last_became_peered":"2024-09-11T08:06:41.620564+0000","last_unstale":"2024-09-11T08:06:53.505247+0000","last_undegraded":"2024-09-11T08:06:53.505247+0000","last_fullsized":"2024-09-11T08:06:53.505247+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:40.591817+0000","last_clean_scrub_stamp":"2024-09-11T08:06:40.591817+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:52:01.769772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:06:53.356073+0000","last_change":"2024-09-11T08:06:43.623022+0000","last_active":"2024-09-11T08:06:53.356073+0000","last_peered":"2024-09-11T08:06:53.356073+0000","last_clean":"2024-09-11T08:06:53.356073+0000","last_became_active":"2024-09-11T08:06:43.622423+0000","last_became_peered":"2024-09-11T08:06:43.622423+0000","last_unstale":"2024-09-11T08:06:53.356073+0000","last_undegraded":"2024-09-11T08:06:53.356073+0000","last_fullsized":"2024-09-11T08:06:53.356073+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:06:42.602559+0000","last_clean_scrub_stamp":"2024-09-11T08:06:42.602559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:50:27.237673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":391,"ondisk_log_size":391,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":77,"num_read_kb":72,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":48,"seq":206158430230,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298600,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446552,"statfs":{"total":95995035648,"available":95689269248,"internally_reserved":0,"allocated":1277952,"data_stored":868928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:07:28 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"last":0.68100000000000005}]},{"osd":1,"last update":"Wed Sep 11 08:07:28 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.54700000000000004}]},{"osd":2,"last update":"Wed Sep 11 08:07:28 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.71199999999999997}]},{"osd":3,"last update":"Wed Sep 11 08:07:28 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.625},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.434}]},{"osd":4,"last update":"Wed Sep 11 08:07:28 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.73199999999999998}]},{"osd":5,"last update":"Wed Sep 11 08:07:28 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.57999999999999996}]},{"osd":6,"last update":"Wed Sep 11 08:07:28 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.60999999999999999}]}]},{"osd":6,"up_from":42,"seq":180388626458,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298576,"kb_used_data":1224,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446576,"statfs":{"total":95995035648,"available":95689293824,"internally_reserved":0,"allocated":1253376,"data_stored":867406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:07:03 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.40999999999999998}]},{"osd":1,"last update":"Wed Sep 11 08:07:03 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.312},{"interface":"front","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.55200000000000005}]},{"osd":2,"last update":"Wed Sep 11 08:07:03 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.45200000000000001}]},{"osd":3,"last update":"Wed Sep 11 08:07:03 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.36699999999999999}]},{"osd":4,"last update":"Wed Sep 11 08:07:03 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.56799999999999995}]},{"osd":5,"last update":"Wed Sep 11 08:07:03 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.432,"15min":0.432},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.433,"5min":0.433,"15min":0.433},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.53200000000000003}]},{"osd":7,"last update":"Wed Sep 11 08:07:28 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.48799999999999999}]}]},{"osd":5,"up_from":36,"seq":154618822688,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297996,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447156,"statfs":{"total":95995035648,"available":95689887744,"internally_reserved":0,"allocated":655360,"data_stored":276659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:07:44 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.51000000000000001,"15min":0.51600000000000001},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.53200000000000003,"15min":0.53600000000000003},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.69199999999999995,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.41499999999999998}]},{"osd":1,"last update":"Wed Sep 11 08:07:44 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.52400000000000002,"15min":0.53100000000000003},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.64800000000000002,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.34399999999999997}]},{"osd":2,"last update":"Wed Sep 11 08:07:44 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.55700000000000005,"15min":0.56499999999999995},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.68000000000000005,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.496,"5min":0.504,"15min":0.505},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.52700000000000002}]},{"osd":3,"last update":"Wed Sep 11 08:07:44 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.67400000000000004,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.434},{"interface":"front","average":{"1min":0.503,"5min":0.54200000000000004,"15min":0.54800000000000004},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.72099999999999997,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.59699999999999998}]},{"osd":4,"last update":"Wed Sep 11 08:07:44 2024","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.36799999999999999,"15min":0.35899999999999999},"min":{"1min":0.29199999999999998,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.39100000000000001,"15min":0.375},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.56000000000000005}]},{"osd":6,"last update":"Wed Sep 11 08:07:05 2024","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.628},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.53900000000000003}]},{"osd":7,"last update":"Wed Sep 11 08:07:27 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"last":0.48999999999999999}]}]},{"osd":4,"up_from":30,"seq":128849018916,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298020,"kb_used_data":664,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447132,"statfs":{"total":95995035648,"available":95689863168,"internally_reserved":0,"allocated":679936,"data_stored":278036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:07:16 2024","interfaces":[{"interface":"back","average":{"1min":1.3169999999999999,"5min":0.68100000000000005,"15min":0.57499999999999996},"min":{"1min":0.35699999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":19.065999999999999,"5min":19.065999999999999,"15min":19.065999999999999},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.51500000000000001,"15min":0.51400000000000001},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.52300000000000002}]},{"osd":1,"last update":"Wed Sep 11 08:07:16 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.503,"15min":0.504},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.68100000000000005,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.53300000000000003,"15min":0.53700000000000003},"min":{"1min":0.39300000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.74199999999999999,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.45200000000000001}]},{"osd":2,"last update":"Wed Sep 11 08:07:16 2024","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":0.67200000000000004,"15min":0.56499999999999995},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":19.041,"5min":19.041,"15min":19.041},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.51500000000000001,"15min":0.51300000000000001},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.63600000000000001}]},{"osd":3,"last update":"Wed Sep 11 08:07:16 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.52900000000000003,"15min":0.52500000000000002},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.505,"15min":0.503},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.71599999999999997,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.54300000000000004}]},{"osd":5,"last update":"Wed Sep 11 08:07:40 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.36299999999999999,"15min":0.35199999999999998},"min":{"1min":0.28000000000000003,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.36599999999999999,"15min":0.35599999999999998},"min":{"1min":0.313,"5min":0.253,"15min":0.253},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.38200000000000001}]},{"osd":6,"last update":"Wed Sep 11 08:07:05 2024","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.46700000000000003}]},{"osd":7,"last update":"Wed Sep 11 08:07:30 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.194,"5min":1.194,"15min":1.194},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.65700000000000003}]}]},{"osd":3,"up_from":25,"seq":107374182441,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298036,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447116,"statfs":{"total":95995035648,"available":95689846784,"internally_reserved":0,"allocated":696320,"data_stored":277526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:07:59 2024","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.37,"15min":0.374},"min":{"1min":0.193,"5min":0.193,"15min":0.193},"max":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.37,"5min":0.39700000000000002,"15min":0.40500000000000003},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.49099999999999999}]},{"osd":1,"last update":"Wed Sep 11 08:07:59 2024","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.40799999999999997,"15min":0.41099999999999998},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.41299999999999998,"15min":0.42699999999999999},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.60399999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.28699999999999998}]},{"osd":2,"last update":"Wed Sep 11 08:07:59 2024","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.44800000000000001,"15min":0.46000000000000002},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.43099999999999999,"15min":0.439},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.307}]},{"osd":4,"last update":"Wed Sep 11 08:07:21 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.379},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.56699999999999995,"15min":0.56100000000000005},"min":{"1min":0.48299999999999998,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.52400000000000002}]},{"osd":5,"last update":"Wed Sep 11 08:07:44 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.58499999999999996,"15min":0.58899999999999997},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.621,"5min":0.60699999999999998,"15min":0.60499999999999998},"min":{"1min":0.42999999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.51400000000000001}]},{"osd":6,"last update":"Wed Sep 11 08:07:05 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.441},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.50700000000000001}]},{"osd":7,"last update":"Wed Sep 11 08:07:28 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.46800000000000003}]}]},{"osd":2,"up_from":18,"seq":77309411373,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298000,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447152,"statfs":{"total":95995035648,"available":95689883648,"internally_reserved":0,"allocated":659456,"data_stored":277165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:07:34 2024","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.35999999999999999,"15min":0.35599999999999998},"min":{"1min":0.255,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"last":0.22700000000000001},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.372,"15min":0.36899999999999999},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.29399999999999998}]},{"osd":1,"last update":"Wed Sep 11 08:07:34 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.39400000000000002,"15min":0.38600000000000001},"min":{"1min":0.27200000000000002,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.311},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.39300000000000002,"15min":0.39300000000000002},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.23999999999999999}]},{"osd":3,"last update":"Wed Sep 11 08:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.40799999999999997,"15min":0.41199999999999998},"min":{"1min":0.26400000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.47999999999999998,"5min":0.626,"15min":0.626},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.44400000000000001,"15min":0.45000000000000001},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.60199999999999998,"5min":0.624,"15min":0.624},"last":0.35599999999999998}]},{"osd":4,"last update":"Wed Sep 11 08:07:20 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.52400000000000002,"15min":0.52700000000000002},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.71499999999999997,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.56499999999999995,"15min":0.56899999999999995},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.68899999999999995,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.43099999999999999}]},{"osd":5,"last update":"Wed Sep 11 08:07:42 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.53300000000000003,"15min":0.53100000000000003},"min":{"1min":0.44800000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.66200000000000003,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.55000000000000004,"15min":0.54800000000000004},"min":{"1min":0.42499999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.68999999999999995,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.51400000000000001}]},{"osd":6,"last update":"Wed Sep 11 08:07:06 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.373},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.42299999999999999}]},{"osd":7,"last update":"Wed Sep 11 08:07:28 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.44}]}]},{"osd":1,"up_from":13,"seq":55834574899,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298052,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447100,"statfs":{"total":95995035648,"available":95689830400,"internally_reserved":0,"allocated":712704,"data_stored":278685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:07:14 2024","interfaces":[{"interface":"back","average":{"1min":0.32300000000000001,"5min":0.32700000000000001,"15min":0.316},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.53600000000000003,"5min":0.60899999999999999,"15min":0.60899999999999999},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.34399999999999997,"15min":0.32400000000000001},"min":{"1min":0.24199999999999999,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.52200000000000002,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.27500000000000002}]},{"osd":2,"last update":"Wed Sep 11 08:07:32 2024","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.501,"5min":0.52200000000000002,"15min":0.52200000000000002},"last":0.38},{"interface":"front","average":{"1min":0.38,"5min":0.39600000000000002,"15min":0.39800000000000002},"min":{"1min":0.30599999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.56000000000000005,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.32400000000000001}]},{"osd":3,"last update":"Wed Sep 11 08:07:00 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.44500000000000001,"15min":0.45100000000000001},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.54400000000000004,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.436,"15min":0.44400000000000001},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.53200000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.309}]},{"osd":4,"last update":"Wed Sep 11 08:07:20 2024","interfaces":[{"interface":"back","average":{"1min":1.419,"5min":0.73799999999999999,"15min":0.624},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":17.425000000000001,"5min":17.425000000000001,"15min":17.425000000000001},"last":0.443},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.56100000000000005,"15min":0.56200000000000006},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.93700000000000006,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.47099999999999997}]},{"osd":5,"last update":"Wed Sep 11 08:07:42 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.79700000000000004,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.60899999999999999,"15min":0.61099999999999999},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.78300000000000003,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.41099999999999998}]},{"osd":6,"last update":"Wed Sep 11 08:07:04 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.51700000000000002},{"interface":"front","average":{"1min":1.403,"5min":1.403,"15min":1.403},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":17.452000000000002,"5min":17.452000000000002,"15min":17.452000000000002},"last":0.42099999999999999}]},{"osd":7,"last update":"Wed Sep 11 08:07:32 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.43099999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738424,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298592,"kb_used_data":1240,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446560,"statfs":{"total":95995035648,"available":95689277440,"internally_reserved":0,"allocated":1269760,"data_stored":868297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 11 08:07:13 2024","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.33400000000000002,"15min":0.33400000000000002},"min":{"1min":0.17199999999999999,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":0.41999999999999998,"5min":0.53400000000000003,"15min":0.53400000000000003},"last":0.26300000000000001},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.35499999999999998,"15min":0.34799999999999998},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.48199999999999998,"5min":0.499,"15min":0.499},"last":0.20100000000000001}]},{"osd":2,"last update":"Wed Sep 11 08:07:30 2024","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.38200000000000001,"15min":0.38800000000000001},"min":{"1min":0.22700000000000001,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.376,"15min":0.38200000000000001},"min":{"1min":0.252,"5min":0.249,"15min":0.249},"max":{"1min":0.496,"5min":0.52700000000000002,"15min":0.52700000000000002},"last":0.245}]},{"osd":3,"last update":"Wed Sep 11 08:07:58 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.41299999999999998,"15min":0.42599999999999999},"min":{"1min":0.28199999999999997,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.54100000000000004,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.31},{"interface":"front","average":{"1min":0.376,"5min":0.39700000000000002,"15min":0.40699999999999997},"min":{"1min":0.222,"5min":0.221,"15min":0.221},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.223}]},{"osd":4,"last update":"Wed Sep 11 08:07:19 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.52000000000000002,"15min":0.52400000000000002},"min":{"1min":0.35499999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55800000000000005,"15min":0.56000000000000005},"min":{"1min":0.44400000000000001,"5min":0.436,"15min":0.436},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.38300000000000001}]},{"osd":5,"last update":"Wed Sep 11 08:07:44 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.55500000000000005,"15min":0.52200000000000002},"min":{"1min":0.41599999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":4.5490000000000004,"5min":4.5490000000000004,"15min":4.5490000000000004},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.40899999999999997,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.40600000000000003}]},{"osd":6,"last update":"Wed Sep 11 08:07:04 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.45500000000000002}]},{"osd":7,"last update":"Wed Sep 11 08:07:28 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.42499999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T08:08:07.914 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-11T08:08:07.915 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-11T08:08:07.915 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-11T08:08:07.915 DEBUG:teuthology.orchestra.run.smithi122:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid f1d75850-7013-11ef-bcea-c7b262605968 -- ceph health --format=json 2024-09-11T08:08:08.161 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:08:08.228 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:08:07 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: ::ffff:172.21.15.135 - - [11/Sep/2024:08:08:07] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:08:08.472 INFO:journalctl@ceph.alertmanager.a.smithi122.stdout:Sep 11 08:08:08 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-alertmanager-a[71755]: ts=2024-09-11T08:08:08.211Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.004608152s 2024-09-11T08:08:08.667 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:08 smithi135 ceph-mon[32689]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:08:08.667 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:08 smithi135 ceph-mon[32689]: from='client.14685 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:08:08.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:08 smithi122 ceph-mon[28992]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:08:08.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:08 smithi122 ceph-mon[28992]: from='client.14685 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:08:08.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:08 smithi122 ceph-mon[39726]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:08:08.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:08 smithi122 ceph-mon[39726]: from='client.14685 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:08:08.978 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:08 smithi135 systemd[1]: Starting Ceph grafana.a for f1d75850-7013-11ef-bcea-c7b262605968... 2024-09-11T08:08:09.478 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:09 smithi135 podman[63259]: 2024-09-11 08:08:08.966616169 +0000 UTC m=+0.015217558 image pull 954c08fa618887fe9a6fe572ae541197c77e625b05f4193df7898f5ac649ada9 quay.io/ceph/ceph-grafana:9.4.7 2024-09-11T08:08:09.478 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:09 smithi135 podman[63259]: 2024-09-11 08:08:09.093470265 +0000 UTC m=+0.142071646 container create 3d910835e226b370d305848077dbaa374ad787eb07a96ff345b4ee31f93aeaaa (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a, vcs-type=git, name=ubi8, com.redhat.component=ubi8-container, distribution-scope=public, io.buildah.version=1.32.0, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, io.openshift.tags=base rhel8, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, version=8.5, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., maintainer=Paul Cuzner , io.k8s.display-name=Red Hat Universal Base Image 8, io.openshift.expose-services=, summary=Grafana Container configured for Ceph mgr/dashboard integration, description=Ceph Grafana Container, build-date=2022-04-29T11:48:11.055897, release=239.1651231664, vendor=Red Hat, Inc., architecture=x86_64) 2024-09-11T08:08:10.029 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T08:08:10.029 INFO:teuthology.orchestra.run.smithi122.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-11T08:08:10.468 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 podman[63259]: 2024-09-11 08:08:10.217683661 +0000 UTC m=+1.266285043 container init 3d910835e226b370d305848077dbaa374ad787eb07a96ff345b4ee31f93aeaaa (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.openshift.expose-services=, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., summary=Grafana Container configured for Ceph mgr/dashboard integration, io.k8s.display-name=Red Hat Universal Base Image 8, build-date=2022-04-29T11:48:11.055897, description=Ceph Grafana Container, distribution-scope=public, vcs-type=git, vendor=Red Hat, Inc., architecture=x86_64, version=8.5, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, io.openshift.tags=base rhel8, com.redhat.component=ubi8-container, name=ubi8, release=239.1651231664, maintainer=Paul Cuzner , io.buildah.version=1.32.0, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com) 2024-09-11T08:08:10.468 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 podman[63259]: 2024-09-11 08:08:10.221393481 +0000 UTC m=+1.269994872 container start 3d910835e226b370d305848077dbaa374ad787eb07a96ff345b4ee31f93aeaaa (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a, vendor=Red Hat, Inc., com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.openshift.expose-services=, com.redhat.component=ubi8-container, io.openshift.tags=base rhel8, description=Ceph Grafana Container, distribution-scope=public, vcs-type=git, architecture=x86_64, release=239.1651231664, io.k8s.display-name=Red Hat Universal Base Image 8, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., build-date=2022-04-29T11:48:11.055897, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, summary=Grafana Container configured for Ceph mgr/dashboard integration, io.buildah.version=1.32.0, name=ubi8, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, version=8.5, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, maintainer=Paul Cuzner ) 2024-09-11T08:08:10.469 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=settings t=2024-09-11T08:08:10.378341559Z level=info msg="Starting Grafana" version=9.4.7 commit=4add91f03d branch=HEAD compiled=2023-03-16T23:56:52Z 2024-09-11T08:08:10.469 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=settings t=2024-09-11T08:08:10.37853304Z level=warn msg="\"sentry\" frontend logging provider is deprecated and will be removed in the next major version. Use \"grafana\" provider instead." 2024-09-11T08:08:10.469 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=settings t=2024-09-11T08:08:10.378552011Z level=info msg="Config loaded from" file=/usr/share/grafana/conf/defaults.ini 2024-09-11T08:08:10.469 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=settings t=2024-09-11T08:08:10.37855907Z level=info msg="Config loaded from" file=/etc/grafana/grafana.ini 2024-09-11T08:08:10.469 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=settings t=2024-09-11T08:08:10.378565312Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_DATA=/var/lib/grafana" 2024-09-11T08:08:10.469 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=settings t=2024-09-11T08:08:10.378571344Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_LOGS=/var/log/grafana" 2024-09-11T08:08:10.470 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=settings t=2024-09-11T08:08:10.378577099Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PLUGINS=/var/lib/grafana/plugins" 2024-09-11T08:08:10.470 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=settings t=2024-09-11T08:08:10.378582966Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PROVISIONING=/etc/grafana/provisioning" 2024-09-11T08:08:10.470 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=settings t=2024-09-11T08:08:10.378588879Z level=info msg="Path Home" path=/usr/share/grafana 2024-09-11T08:08:10.470 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=settings t=2024-09-11T08:08:10.378594713Z level=info msg="Path Data" path=/var/lib/grafana 2024-09-11T08:08:10.470 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=settings t=2024-09-11T08:08:10.378601873Z level=info msg="Path Logs" path=/var/log/grafana 2024-09-11T08:08:10.470 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=settings t=2024-09-11T08:08:10.378607822Z level=info msg="Path Plugins" path=/var/lib/grafana/plugins 2024-09-11T08:08:10.470 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=settings t=2024-09-11T08:08:10.378613955Z level=info msg="Path Provisioning" path=/etc/grafana/provisioning 2024-09-11T08:08:10.470 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=settings t=2024-09-11T08:08:10.378619883Z level=info msg="App mode production" 2024-09-11T08:08:10.470 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=sqlstore t=2024-09-11T08:08:10.378662468Z level=info msg="Connecting to DB" dbtype=sqlite3 2024-09-11T08:08:10.470 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=sqlstore t=2024-09-11T08:08:10.378678284Z level=warn msg="SQLite database file has broader permissions than it should" path=/var/lib/grafana/grafana.db mode=-rw-r--r-- expected=-rw-r----- 2024-09-11T08:08:10.470 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.379554132Z level=info msg="Starting DB migrations" 2024-09-11T08:08:10.470 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.37963367Z level=info msg="Executing migration" id="create migration_log table" 2024-09-11T08:08:10.471 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.380154551Z level=info msg="Executing migration" id="create user table" 2024-09-11T08:08:10.471 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.38063328Z level=info msg="Executing migration" id="add unique index user.login" 2024-09-11T08:08:10.471 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.381114875Z level=info msg="Executing migration" id="add unique index user.email" 2024-09-11T08:08:10.471 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.381541531Z level=info msg="Executing migration" id="drop index UQE_user_login - v1" 2024-09-11T08:08:10.471 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.382144792Z level=info msg="Executing migration" id="drop index UQE_user_email - v1" 2024-09-11T08:08:10.471 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.382525896Z level=info msg="Executing migration" id="Rename table user to user_v1 - v1" 2024-09-11T08:08:10.471 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.38323586Z level=info msg="Executing migration" id="create user table v2" 2024-09-11T08:08:10.471 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.383629231Z level=info msg="Executing migration" id="create index UQE_user_login - v2" 2024-09-11T08:08:10.471 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.383983318Z level=info msg="Executing migration" id="create index UQE_user_email - v2" 2024-09-11T08:08:10.471 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.38441344Z level=info msg="Executing migration" id="copy data_source v1 to v2" 2024-09-11T08:08:10.471 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.384730687Z level=info msg="Executing migration" id="Drop old table user_v1" 2024-09-11T08:08:10.471 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.385059306Z level=info msg="Executing migration" id="Add column help_flags1 to user table" 2024-09-11T08:08:10.471 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.385474186Z level=info msg="Executing migration" id="Update user table charset" 2024-09-11T08:08:10.472 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.385740041Z level=info msg="Executing migration" id="Add last_seen_at column to user" 2024-09-11T08:08:10.472 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.38615054Z level=info msg="Executing migration" id="Add missing user data" 2024-09-11T08:08:10.472 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.386606116Z level=info msg="Executing migration" id="Add is_disabled column to user" 2024-09-11T08:08:10.472 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.387238498Z level=info msg="Executing migration" id="Add index user.login/user.email" 2024-09-11T08:08:10.472 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.387656505Z level=info msg="Executing migration" id="Add is_service_account column to user" 2024-09-11T08:08:10.472 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.388038555Z level=info msg="Executing migration" id="Update is_service_account column to nullable" 2024-09-11T08:08:10.472 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.389129749Z level=info msg="Executing migration" id="create temp user table v1-7" 2024-09-11T08:08:10.472 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.389553065Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v1-7" 2024-09-11T08:08:10.472 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.389986859Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v1-7" 2024-09-11T08:08:10.472 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.390422356Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v1-7" 2024-09-11T08:08:10.472 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.390826102Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v1-7" 2024-09-11T08:08:10.472 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.391223545Z level=info msg="Executing migration" id="Update temp_user table charset" 2024-09-11T08:08:10.472 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.391513779Z level=info msg="Executing migration" id="drop index IDX_temp_user_email - v1" 2024-09-11T08:08:10.473 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.391862752Z level=info msg="Executing migration" id="drop index IDX_temp_user_org_id - v1" 2024-09-11T08:08:10.473 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.392252214Z level=info msg="Executing migration" id="drop index IDX_temp_user_code - v1" 2024-09-11T08:08:10.473 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.392639322Z level=info msg="Executing migration" id="drop index IDX_temp_user_status - v1" 2024-09-11T08:08:10.473 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.393016818Z level=info msg="Executing migration" id="Rename table temp_user to temp_user_tmp_qwerty - v1" 2024-09-11T08:08:10.473 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.393660316Z level=info msg="Executing migration" id="create temp_user v2" 2024-09-11T08:08:10.473 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.394041506Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v2" 2024-09-11T08:08:10.473 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.394448719Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v2" 2024-09-11T08:08:10.473 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.394820112Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v2" 2024-09-11T08:08:10.473 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.39521684Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v2" 2024-09-11T08:08:10.473 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.395611944Z level=info msg="Executing migration" id="copy temp_user v1 to v2" 2024-09-11T08:08:10.473 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.395923581Z level=info msg="Executing migration" id="drop temp_user_tmp_qwerty" 2024-09-11T08:08:10.473 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.396274858Z level=info msg="Executing migration" id="Set created for temp users that will otherwise prematurely expire" 2024-09-11T08:08:10.473 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.396613923Z level=info msg="Executing migration" id="create star table" 2024-09-11T08:08:10.474 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.39695448Z level=info msg="Executing migration" id="add unique index star.user_id_dashboard_id" 2024-09-11T08:08:10.474 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.397371661Z level=info msg="Executing migration" id="create org table v1" 2024-09-11T08:08:10.474 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.397761939Z level=info msg="Executing migration" id="create index UQE_org_name - v1" 2024-09-11T08:08:10.474 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.398171103Z level=info msg="Executing migration" id="create org_user table v1" 2024-09-11T08:08:10.474 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.398534487Z level=info msg="Executing migration" id="create index IDX_org_user_org_id - v1" 2024-09-11T08:08:10.474 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.398921637Z level=info msg="Executing migration" id="create index UQE_org_user_org_id_user_id - v1" 2024-09-11T08:08:10.475 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.399326282Z level=info msg="Executing migration" id="create index IDX_org_user_user_id - v1" 2024-09-11T08:08:10.475 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.399724619Z level=info msg="Executing migration" id="Update org table charset" 2024-09-11T08:08:10.475 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.400006847Z level=info msg="Executing migration" id="Update org_user table charset" 2024-09-11T08:08:10.475 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.400313514Z level=info msg="Executing migration" id="Migrate all Read Only Viewers to Viewers" 2024-09-11T08:08:10.475 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.400822524Z level=info msg="Executing migration" id="create dashboard table" 2024-09-11T08:08:10.475 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.401233635Z level=info msg="Executing migration" id="add index dashboard.account_id" 2024-09-11T08:08:10.475 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.403800684Z level=info msg="Executing migration" id="add unique index dashboard_account_id_slug" 2024-09-11T08:08:10.475 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.406469556Z level=info msg="Executing migration" id="create dashboard_tag table" 2024-09-11T08:08:10.475 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.408983006Z level=info msg="Executing migration" id="add unique index dashboard_tag.dasboard_id_term" 2024-09-11T08:08:10.475 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.411873002Z level=info msg="Executing migration" id="drop index UQE_dashboard_tag_dashboard_id_term - v1" 2024-09-11T08:08:10.475 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.414455274Z level=info msg="Executing migration" id="Rename table dashboard to dashboard_v1 - v1" 2024-09-11T08:08:10.476 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.417064041Z level=info msg="Executing migration" id="create dashboard v2" 2024-09-11T08:08:10.476 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.417471398Z level=info msg="Executing migration" id="create index IDX_dashboard_org_id - v2" 2024-09-11T08:08:10.476 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.420192725Z level=info msg="Executing migration" id="create index UQE_dashboard_org_id_slug - v2" 2024-09-11T08:08:10.476 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.422874122Z level=info msg="Executing migration" id="copy dashboard v1 to v2" 2024-09-11T08:08:10.476 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.425489795Z level=info msg="Executing migration" id="drop table dashboard_v1" 2024-09-11T08:08:10.476 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.426067641Z level=info msg="Executing migration" id="alter dashboard.data to mediumtext v1" 2024-09-11T08:08:10.476 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.428858205Z level=info msg="Executing migration" id="Add column updated_by in dashboard - v2" 2024-09-11T08:08:10.476 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.431788897Z level=info msg="Executing migration" id="Add column created_by in dashboard - v2" 2024-09-11T08:08:10.476 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.434440894Z level=info msg="Executing migration" id="Add column gnetId in dashboard" 2024-09-11T08:08:10.476 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.435115324Z level=info msg="Executing migration" id="Add index for gnetId in dashboard" 2024-09-11T08:08:10.476 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.437642062Z level=info msg="Executing migration" id="Add column plugin_id in dashboard" 2024-09-11T08:08:10.476 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.440264611Z level=info msg="Executing migration" id="Add index for plugin_id in dashboard" 2024-09-11T08:08:10.477 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.442879348Z level=info msg="Executing migration" id="Add index for dashboard_id in dashboard_tag" 2024-09-11T08:08:10.477 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.443311819Z level=info msg="Executing migration" id="Update dashboard table charset" 2024-09-11T08:08:10.477 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.446078357Z level=info msg="Executing migration" id="Update dashboard_tag table charset" 2024-09-11T08:08:10.477 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.448775193Z level=info msg="Executing migration" id="Add column folder_id in dashboard" 2024-09-11T08:08:10.477 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.449358777Z level=info msg="Executing migration" id="Add column isFolder in dashboard" 2024-09-11T08:08:10.477 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.451422826Z level=info msg="Executing migration" id="Add column has_acl in dashboard" 2024-09-11T08:08:10.477 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.454247158Z level=info msg="Executing migration" id="Add column uid in dashboard" 2024-09-11T08:08:10.477 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.456569697Z level=info msg="Executing migration" id="Update uid column values in dashboard" 2024-09-11T08:08:10.477 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.459206664Z level=info msg="Executing migration" id="Add unique index dashboard_org_id_uid" 2024-09-11T08:08:10.477 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.459648362Z level=info msg="Executing migration" id="Remove unique index org_id_slug" 2024-09-11T08:08:10.477 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.46048764Z level=info msg="Executing migration" id="Update dashboard title length" 2024-09-11T08:08:10.477 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.460812791Z level=info msg="Executing migration" id="Add unique index for dashboard_org_id_title_folder_id" 2024-09-11T08:08:10.478 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.461392944Z level=info msg="Executing migration" id="create dashboard_provisioning" 2024-09-11T08:08:10.478 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.462205891Z level=info msg="Executing migration" id="Rename table dashboard_provisioning to dashboard_provisioning_tmp_qwerty - v1" 2024-09-11T08:08:10.478 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.463107223Z level=info msg="Executing migration" id="create dashboard_provisioning v2" 2024-09-11T08:08:10.478 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.463921313Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id - v2" 2024-09-11T08:08:10.478 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.464918477Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id_name - v2" 2024-09-11T08:08:10.478 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.467545654Z level=info msg="Executing migration" id="copy dashboard_provisioning v1 to v2" 2024-09-11T08:08:10.478 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.467901412Z level=info msg="Executing migration" id="drop dashboard_provisioning_tmp_qwerty" 2024-09-11T08:08:10.631 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:10 smithi122 ceph-mon[28992]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:08:10.631 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:10 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/2536133640' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-11T08:08:10.632 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:10 smithi122 ceph-mon[39726]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:08:10.632 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:10 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/2536133640' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-11T08:08:10.657 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-11T08:08:10.657 INFO:tasks.cephadm:Setup complete, yielding 2024-09-11T08:08:10.658 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-11T08:08:11.174 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi122.front.sepia.ceph.com 2024-09-11T08:08:11.175 DEBUG:teuthology.orchestra.run.smithi122:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f1d75850-7013-11ef-bcea-c7b262605968 -- bash -c 'set -ex 2024-09-11T08:08:11.175 DEBUG:teuthology.orchestra.run.smithi122:> for f in osd.0 osd.1 osd.2 osd.3 osd.4 osd.5 osd.6 osd.7 mgr.y mgr.x 2024-09-11T08:08:11.175 DEBUG:teuthology.orchestra.run.smithi122:> do 2024-09-11T08:08:11.176 DEBUG:teuthology.orchestra.run.smithi122:> echo "rotating key for $f" 2024-09-11T08:08:11.176 DEBUG:teuthology.orchestra.run.smithi122:> K=$(ceph auth get-key $f) 2024-09-11T08:08:11.176 DEBUG:teuthology.orchestra.run.smithi122:> NK="$K" 2024-09-11T08:08:11.176 DEBUG:teuthology.orchestra.run.smithi122:> ceph orch daemon rotate-key $f 2024-09-11T08:08:11.176 DEBUG:teuthology.orchestra.run.smithi122:> while [ "$K" == "$NK" ]; do 2024-09-11T08:08:11.176 DEBUG:teuthology.orchestra.run.smithi122:> sleep 5 2024-09-11T08:08:11.176 DEBUG:teuthology.orchestra.run.smithi122:> NK=$(ceph auth get-key $f) 2024-09-11T08:08:11.176 DEBUG:teuthology.orchestra.run.smithi122:> done 2024-09-11T08:08:11.176 DEBUG:teuthology.orchestra.run.smithi122:> done 2024-09-11T08:08:11.177 DEBUG:teuthology.orchestra.run.smithi122:> ' 2024-09-11T08:08:11.179 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-mon[32689]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:08:11.179 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/2536133640' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-11T08:08:11.183 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.473828595Z level=info msg="Executing migration" id="Add check_sum column" 2024-09-11T08:08:11.183 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.474418648Z level=info msg="Executing migration" id="Add index for dashboard_title" 2024-09-11T08:08:11.184 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.477265688Z level=info msg="Executing migration" id="delete tags for deleted dashboards" 2024-09-11T08:08:11.184 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.479809463Z level=info msg="Executing migration" id="delete stars for deleted dashboards" 2024-09-11T08:08:11.184 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.482749453Z level=info msg="Executing migration" id="Add index for dashboard_is_folder" 2024-09-11T08:08:11.184 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.485662979Z level=info msg="Executing migration" id="Add isPublic for dashboard" 2024-09-11T08:08:11.184 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.488572981Z level=info msg="Executing migration" id="create data_source table" 2024-09-11T08:08:11.185 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.492118067Z level=info msg="Executing migration" id="add index data_source.account_id" 2024-09-11T08:08:11.185 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.497049159Z level=info msg="Executing migration" id="add unique index data_source.account_id_name" 2024-09-11T08:08:11.185 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.49985669Z level=info msg="Executing migration" id="drop index IDX_data_source_account_id - v1" 2024-09-11T08:08:11.185 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.502781897Z level=info msg="Executing migration" id="drop index UQE_data_source_account_id_name - v1" 2024-09-11T08:08:11.185 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.505795149Z level=info msg="Executing migration" id="Rename table data_source to data_source_v1 - v1" 2024-09-11T08:08:11.185 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.508150785Z level=info msg="Executing migration" id="create data_source table v2" 2024-09-11T08:08:11.185 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.510989862Z level=info msg="Executing migration" id="create index IDX_data_source_org_id - v2" 2024-09-11T08:08:11.185 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.513807442Z level=info msg="Executing migration" id="create index UQE_data_source_org_id_name - v2" 2024-09-11T08:08:11.185 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.516749145Z level=info msg="Executing migration" id="Drop old table data_source_v1 #2" 2024-09-11T08:08:11.185 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.519337354Z level=info msg="Executing migration" id="Add column with_credentials" 2024-09-11T08:08:11.185 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.521941868Z level=info msg="Executing migration" id="Add secure json data column" 2024-09-11T08:08:11.186 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.524671828Z level=info msg="Executing migration" id="Update data_source table charset" 2024-09-11T08:08:11.186 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.527361335Z level=info msg="Executing migration" id="Update initial version to 1" 2024-09-11T08:08:11.186 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.529983933Z level=info msg="Executing migration" id="Add read_only data column" 2024-09-11T08:08:11.186 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.532247113Z level=info msg="Executing migration" id="Migrate logging ds to loki ds" 2024-09-11T08:08:11.186 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.535042749Z level=info msg="Executing migration" id="Update json_data with nulls" 2024-09-11T08:08:11.186 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.537655283Z level=info msg="Executing migration" id="Add uid column" 2024-09-11T08:08:11.186 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.54055187Z level=info msg="Executing migration" id="Update uid value" 2024-09-11T08:08:11.186 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.542967689Z level=info msg="Executing migration" id="Add unique index datasource_org_id_uid" 2024-09-11T08:08:11.186 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.545627359Z level=info msg="Executing migration" id="add unique index datasource_org_id_is_default" 2024-09-11T08:08:11.186 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.548218579Z level=info msg="Executing migration" id="create api_key table" 2024-09-11T08:08:11.186 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.550983336Z level=info msg="Executing migration" id="add index api_key.account_id" 2024-09-11T08:08:11.187 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.553661653Z level=info msg="Executing migration" id="add index api_key.key" 2024-09-11T08:08:11.187 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.556429445Z level=info msg="Executing migration" id="add index api_key.account_id_name" 2024-09-11T08:08:11.187 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.558856283Z level=info msg="Executing migration" id="drop index IDX_api_key_account_id - v1" 2024-09-11T08:08:11.187 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.561753806Z level=info msg="Executing migration" id="drop index UQE_api_key_key - v1" 2024-09-11T08:08:11.187 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.562421027Z level=info msg="Executing migration" id="drop index UQE_api_key_account_id_name - v1" 2024-09-11T08:08:11.187 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.562841754Z level=info msg="Executing migration" id="Rename table api_key to api_key_v1 - v1" 2024-09-11T08:08:11.187 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.563902117Z level=info msg="Executing migration" id="create api_key table v2" 2024-09-11T08:08:11.187 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.564754182Z level=info msg="Executing migration" id="create index IDX_api_key_org_id - v2" 2024-09-11T08:08:11.187 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.565562471Z level=info msg="Executing migration" id="create index UQE_api_key_key - v2" 2024-09-11T08:08:11.187 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.566007233Z level=info msg="Executing migration" id="create index UQE_api_key_org_id_name - v2" 2024-09-11T08:08:11.187 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.566989003Z level=info msg="Executing migration" id="copy api_key v1 to v2" 2024-09-11T08:08:11.187 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.567806089Z level=info msg="Executing migration" id="Drop old table api_key_v1" 2024-09-11T08:08:11.187 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.568474292Z level=info msg="Executing migration" id="Update api_key table charset" 2024-09-11T08:08:11.188 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.568758191Z level=info msg="Executing migration" id="Add expires to api_key table" 2024-09-11T08:08:11.188 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.569612886Z level=info msg="Executing migration" id="Add service account foreign key" 2024-09-11T08:08:11.188 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.570452179Z level=info msg="Executing migration" id="set service account foreign key to nil if 0" 2024-09-11T08:08:11.188 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.571378141Z level=info msg="Executing migration" id="Add last_used_at to api_key table" 2024-09-11T08:08:11.188 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.572026734Z level=info msg="Executing migration" id="Add is_revoked column to api_key table" 2024-09-11T08:08:11.188 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.572811952Z level=info msg="Executing migration" id="create dashboard_snapshot table v4" 2024-09-11T08:08:11.188 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.573613935Z level=info msg="Executing migration" id="drop table dashboard_snapshot_v4 #1" 2024-09-11T08:08:11.188 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.574476306Z level=info msg="Executing migration" id="create dashboard_snapshot table v5 #2" 2024-09-11T08:08:11.188 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.575441031Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_key - v5" 2024-09-11T08:08:11.188 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.576103253Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_delete_key - v5" 2024-09-11T08:08:11.188 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.576966888Z level=info msg="Executing migration" id="create index IDX_dashboard_snapshot_user_id - v5" 2024-09-11T08:08:11.188 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.577829597Z level=info msg="Executing migration" id="alter dashboard_snapshot to mediumtext v2" 2024-09-11T08:08:11.188 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.578138403Z level=info msg="Executing migration" id="Update dashboard_snapshot table charset" 2024-09-11T08:08:11.189 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.579107548Z level=info msg="Executing migration" id="Add column external_delete_url to dashboard_snapshots table" 2024-09-11T08:08:11.189 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.579862271Z level=info msg="Executing migration" id="Add encrypted dashboard json column" 2024-09-11T08:08:11.189 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.580833564Z level=info msg="Executing migration" id="Change dashboard_encrypted column to MEDIUMBLOB" 2024-09-11T08:08:11.189 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.581302252Z level=info msg="Executing migration" id="create quota table v1" 2024-09-11T08:08:11.189 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.582252266Z level=info msg="Executing migration" id="create index UQE_quota_org_id_user_id_target - v1" 2024-09-11T08:08:11.189 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.582893721Z level=info msg="Executing migration" id="Update quota table charset" 2024-09-11T08:08:11.189 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.583742423Z level=info msg="Executing migration" id="create plugin_setting table" 2024-09-11T08:08:11.189 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.584156911Z level=info msg="Executing migration" id="create index UQE_plugin_setting_org_id_plugin_id - v1" 2024-09-11T08:08:11.189 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.585083807Z level=info msg="Executing migration" id="Add column plugin_version to plugin_settings" 2024-09-11T08:08:11.189 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.585810273Z level=info msg="Executing migration" id="Update plugin_setting table charset" 2024-09-11T08:08:11.190 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.586716294Z level=info msg="Executing migration" id="create session table" 2024-09-11T08:08:11.190 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.587264305Z level=info msg="Executing migration" id="Drop old table playlist table" 2024-09-11T08:08:11.190 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.587561956Z level=info msg="Executing migration" id="Drop old table playlist_item table" 2024-09-11T08:08:11.190 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.590194772Z level=info msg="Executing migration" id="create playlist table v2" 2024-09-11T08:08:11.190 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.590574705Z level=info msg="Executing migration" id="create playlist item table v2" 2024-09-11T08:08:11.190 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.593072064Z level=info msg="Executing migration" id="Update playlist table charset" 2024-09-11T08:08:11.190 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.596055029Z level=info msg="Executing migration" id="Update playlist_item table charset" 2024-09-11T08:08:11.190 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.596374603Z level=info msg="Executing migration" id="drop preferences table v2" 2024-09-11T08:08:11.190 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.598971814Z level=info msg="Executing migration" id="drop preferences table v3" 2024-09-11T08:08:11.190 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.599406274Z level=info msg="Executing migration" id="create preferences table v3" 2024-09-11T08:08:11.190 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.601985048Z level=info msg="Executing migration" id="Update preferences table charset" 2024-09-11T08:08:11.190 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.60471916Z level=info msg="Executing migration" id="Add column team_id in preferences" 2024-09-11T08:08:11.190 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.605359942Z level=info msg="Executing migration" id="Update team_id column values in preferences" 2024-09-11T08:08:11.191 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.608357857Z level=info msg="Executing migration" id="Add column week_start in preferences" 2024-09-11T08:08:11.191 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.611284474Z level=info msg="Executing migration" id="Add column preferences.json_data" 2024-09-11T08:08:11.191 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.613900654Z level=info msg="Executing migration" id="alter preferences.json_data to mediumtext v1" 2024-09-11T08:08:11.191 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.614261999Z level=info msg="Executing migration" id="Add preferences index org_id" 2024-09-11T08:08:11.191 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.616971931Z level=info msg="Executing migration" id="Add preferences index user_id" 2024-09-11T08:08:11.191 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.619804449Z level=info msg="Executing migration" id="create alert table v1" 2024-09-11T08:08:11.191 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.622470127Z level=info msg="Executing migration" id="add index alert org_id & id " 2024-09-11T08:08:11.191 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.625165145Z level=info msg="Executing migration" id="add index alert state" 2024-09-11T08:08:11.191 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.627638511Z level=info msg="Executing migration" id="add index alert dashboard_id" 2024-09-11T08:08:11.191 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.630334006Z level=info msg="Executing migration" id="Create alert_rule_tag table v1" 2024-09-11T08:08:11.191 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.632727091Z level=info msg="Executing migration" id="Add unique index alert_rule_tag.alert_id_tag_id" 2024-09-11T08:08:11.191 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.63314421Z level=info msg="Executing migration" id="drop index UQE_alert_rule_tag_alert_id_tag_id - v1" 2024-09-11T08:08:11.191 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.635550211Z level=info msg="Executing migration" id="Rename table alert_rule_tag to alert_rule_tag_v1 - v1" 2024-09-11T08:08:11.192 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.638327464Z level=info msg="Executing migration" id="Create alert_rule_tag table v2" 2024-09-11T08:08:11.192 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.640901318Z level=info msg="Executing migration" id="create index UQE_alert_rule_tag_alert_id_tag_id - Add unique index alert_rule_tag.alert_id_tag_id V2" 2024-09-11T08:08:11.192 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.641343967Z level=info msg="Executing migration" id="copy alert_rule_tag v1 to v2" 2024-09-11T08:08:11.192 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.64387179Z level=info msg="Executing migration" id="drop table alert_rule_tag_v1" 2024-09-11T08:08:11.192 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.646422677Z level=info msg="Executing migration" id="create alert_notification table v1" 2024-09-11T08:08:11.192 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.649015268Z level=info msg="Executing migration" id="Add column is_default" 2024-09-11T08:08:11.192 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.649669064Z level=info msg="Executing migration" id="Add column frequency" 2024-09-11T08:08:11.192 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.652539367Z level=info msg="Executing migration" id="Add column send_reminder" 2024-09-11T08:08:11.192 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.654872102Z level=info msg="Executing migration" id="Add column disable_resolve_message" 2024-09-11T08:08:11.192 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.657531198Z level=info msg="Executing migration" id="add index alert_notification org_id & name" 2024-09-11T08:08:11.192 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.657958892Z level=info msg="Executing migration" id="Update alert table charset" 2024-09-11T08:08:11.193 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.660669457Z level=info msg="Executing migration" id="Update alert_notification table charset" 2024-09-11T08:08:11.193 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.661484853Z level=info msg="Executing migration" id="create notification_journal table v1" 2024-09-11T08:08:11.193 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.662269795Z level=info msg="Executing migration" id="add index notification_journal org_id & alert_id & notifier_id" 2024-09-11T08:08:11.193 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.662698104Z level=info msg="Executing migration" id="drop alert_notification_journal" 2024-09-11T08:08:11.193 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.663497843Z level=info msg="Executing migration" id="create alert_notification_state table v1" 2024-09-11T08:08:11.193 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.663965629Z level=info msg="Executing migration" id="add index alert_notification_state org_id & alert_id & notifier_id" 2024-09-11T08:08:11.193 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.664428712Z level=info msg="Executing migration" id="Add for to alert table" 2024-09-11T08:08:11.193 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.665370791Z level=info msg="Executing migration" id="Add column uid in alert_notification" 2024-09-11T08:08:11.193 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.666196696Z level=info msg="Executing migration" id="Update uid column values in alert_notification" 2024-09-11T08:08:11.193 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.666554238Z level=info msg="Executing migration" id="Add unique index alert_notification_org_id_uid" 2024-09-11T08:08:11.193 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.667308374Z level=info msg="Executing migration" id="Remove unique index org_id_name" 2024-09-11T08:08:11.193 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.668091862Z level=info msg="Executing migration" id="Add column secure_settings in alert_notification" 2024-09-11T08:08:11.193 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.668935994Z level=info msg="Executing migration" id="alter alert.settings to mediumtext" 2024-09-11T08:08:11.194 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.669260857Z level=info msg="Executing migration" id="Add non-unique index alert_notification_state_alert_id" 2024-09-11T08:08:11.194 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.672050135Z level=info msg="Executing migration" id="Add non-unique index alert_rule_tag_alert_id" 2024-09-11T08:08:11.194 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.672674732Z level=info msg="Executing migration" id="Drop old annotation table v4" 2024-09-11T08:08:11.194 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.674889691Z level=info msg="Executing migration" id="create annotation table v5" 2024-09-11T08:08:11.194 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.675365239Z level=info msg="Executing migration" id="add index annotation 0 v3" 2024-09-11T08:08:11.194 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.677637195Z level=info msg="Executing migration" id="add index annotation 1 v3" 2024-09-11T08:08:11.194 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.67809376Z level=info msg="Executing migration" id="add index annotation 2 v3" 2024-09-11T08:08:11.194 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.680608551Z level=info msg="Executing migration" id="add index annotation 3 v3" 2024-09-11T08:08:11.194 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.683112114Z level=info msg="Executing migration" id="add index annotation 4 v3" 2024-09-11T08:08:11.194 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.685856825Z level=info msg="Executing migration" id="Update annotation table charset" 2024-09-11T08:08:11.194 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.686159929Z level=info msg="Executing migration" id="Add column region_id to annotation table" 2024-09-11T08:08:11.194 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.688755199Z level=info msg="Executing migration" id="Drop category_id index" 2024-09-11T08:08:11.194 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.691518262Z level=info msg="Executing migration" id="Add column tags to annotation table" 2024-09-11T08:08:11.195 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.694258198Z level=info msg="Executing migration" id="Create annotation_tag table v2" 2024-09-11T08:08:11.195 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.694765517Z level=info msg="Executing migration" id="Add unique index annotation_tag.annotation_id_tag_id" 2024-09-11T08:08:11.195 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.69798851Z level=info msg="Executing migration" id="drop index UQE_annotation_tag_annotation_id_tag_id - v2" 2024-09-11T08:08:11.195 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.6998987Z level=info msg="Executing migration" id="Rename table annotation_tag to annotation_tag_v2 - v2" 2024-09-11T08:08:11.195 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.702527585Z level=info msg="Executing migration" id="Create annotation_tag table v3" 2024-09-11T08:08:11.195 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.702960546Z level=info msg="Executing migration" id="create index UQE_annotation_tag_annotation_id_tag_id - Add unique index annotation_tag.annotation_id_tag_id V3" 2024-09-11T08:08:11.195 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.705494259Z level=info msg="Executing migration" id="copy annotation_tag v2 to v3" 2024-09-11T08:08:11.195 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.707952119Z level=info msg="Executing migration" id="drop table annotation_tag_v2" 2024-09-11T08:08:11.195 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.710789106Z level=info msg="Executing migration" id="Update alert annotations and set TEXT to empty" 2024-09-11T08:08:11.195 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.711191122Z level=info msg="Executing migration" id="Add created time to annotation table" 2024-09-11T08:08:11.195 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.711970712Z level=info msg="Executing migration" id="Add updated time to annotation table" 2024-09-11T08:08:11.195 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.712789657Z level=info msg="Executing migration" id="Add index for created in annotation table" 2024-09-11T08:08:11.195 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.713675309Z level=info msg="Executing migration" id="Add index for updated in annotation table" 2024-09-11T08:08:11.196 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.714358363Z level=info msg="Executing migration" id="Convert existing annotations from seconds to milliseconds" 2024-09-11T08:08:11.196 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.715251712Z level=info msg="Executing migration" id="Add epoch_end column" 2024-09-11T08:08:11.196 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.716016369Z level=info msg="Executing migration" id="Add index for epoch_end" 2024-09-11T08:08:11.196 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.716789196Z level=info msg="Executing migration" id="Make epoch_end the same as epoch" 2024-09-11T08:08:11.196 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.71708675Z level=info msg="Executing migration" id="Move region to single row" 2024-09-11T08:08:11.196 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.717462511Z level=info msg="Executing migration" id="Remove index org_id_epoch from annotation table" 2024-09-11T08:08:11.196 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.717902354Z level=info msg="Executing migration" id="Remove index org_id_dashboard_id_panel_id_epoch from annotation table" 2024-09-11T08:08:11.196 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.718329191Z level=info msg="Executing migration" id="Add index for org_id_dashboard_id_epoch_end_epoch on annotation table" 2024-09-11T08:08:11.196 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.718782973Z level=info msg="Executing migration" id="Add index for org_id_epoch_end_epoch on annotation table" 2024-09-11T08:08:11.196 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.719626768Z level=info msg="Executing migration" id="Remove index org_id_epoch_epoch_end from annotation table" 2024-09-11T08:08:11.196 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.720047021Z level=info msg="Executing migration" id="Add index for alert_id on annotation table" 2024-09-11T08:08:11.196 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.720850442Z level=info msg="Executing migration" id="Increase tags column to length 4096" 2024-09-11T08:08:11.196 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.721640936Z level=info msg="Executing migration" id="create test_data table" 2024-09-11T08:08:11.197 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.722048546Z level=info msg="Executing migration" id="create dashboard_version table v1" 2024-09-11T08:08:11.197 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.722437913Z level=info msg="Executing migration" id="add index dashboard_version.dashboard_id" 2024-09-11T08:08:11.197 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.723402556Z level=info msg="Executing migration" id="add unique index dashboard_version.dashboard_id and dashboard_version.version" 2024-09-11T08:08:11.197 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.724122648Z level=info msg="Executing migration" id="Set dashboard version to 1 where 0" 2024-09-11T08:08:11.197 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.724908632Z level=info msg="Executing migration" id="save existing dashboard data in dashboard_version table v1" 2024-09-11T08:08:11.197 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.725804705Z level=info msg="Executing migration" id="alter dashboard_version.data to mediumtext v1" 2024-09-11T08:08:11.197 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.726666391Z level=info msg="Executing migration" id="create team table" 2024-09-11T08:08:11.197 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.727386602Z level=info msg="Executing migration" id="add index team.org_id" 2024-09-11T08:08:11.197 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.728173188Z level=info msg="Executing migration" id="add unique index team_org_id_name" 2024-09-11T08:08:11.197 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.728589178Z level=info msg="Executing migration" id="create team member table" 2024-09-11T08:08:11.197 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.729585833Z level=info msg="Executing migration" id="add index team_member.org_id" 2024-09-11T08:08:11.197 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.730256045Z level=info msg="Executing migration" id="add unique index team_member_org_id_team_id_user_id" 2024-09-11T08:08:11.198 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.731107272Z level=info msg="Executing migration" id="add index team_member.team_id" 2024-09-11T08:08:11.198 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.73154189Z level=info msg="Executing migration" id="Add column email to team table" 2024-09-11T08:08:11.198 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.732316202Z level=info msg="Executing migration" id="Add column external to team_member table" 2024-09-11T08:08:11.198 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.733166382Z level=info msg="Executing migration" id="Add column permission to team_member table" 2024-09-11T08:08:11.198 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.733990712Z level=info msg="Executing migration" id="create dashboard acl table" 2024-09-11T08:08:11.198 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.734401468Z level=info msg="Executing migration" id="add index dashboard_acl_dashboard_id" 2024-09-11T08:08:11.198 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.735259659Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_user_id" 2024-09-11T08:08:11.199 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.738003716Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_team_id" 2024-09-11T08:08:11.199 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.738428713Z level=info msg="Executing migration" id="add index dashboard_acl_user_id" 2024-09-11T08:08:11.199 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.738881345Z level=info msg="Executing migration" id="add index dashboard_acl_team_id" 2024-09-11T08:08:11.199 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.741494077Z level=info msg="Executing migration" id="add index dashboard_acl_org_id_role" 2024-09-11T08:08:11.199 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.742042914Z level=info msg="Executing migration" id="add index dashboard_permission" 2024-09-11T08:08:11.199 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.742450033Z level=info msg="Executing migration" id="save default acl rules in dashboard_acl table" 2024-09-11T08:08:11.199 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.742808656Z level=info msg="Executing migration" id="delete acl rules for deleted dashboards and folders" 2024-09-11T08:08:11.199 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.743078175Z level=info msg="Executing migration" id="create tag table" 2024-09-11T08:08:11.199 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.743487838Z level=info msg="Executing migration" id="add index tag.key_value" 2024-09-11T08:08:11.199 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.743891767Z level=info msg="Executing migration" id="create login attempt table" 2024-09-11T08:08:11.199 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.744280097Z level=info msg="Executing migration" id="add index login_attempt.username" 2024-09-11T08:08:11.200 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.74468596Z level=info msg="Executing migration" id="drop index IDX_login_attempt_username - v1" 2024-09-11T08:08:11.200 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.745059063Z level=info msg="Executing migration" id="Rename table login_attempt to login_attempt_tmp_qwerty - v1" 2024-09-11T08:08:11.200 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.746905029Z level=info msg="Executing migration" id="create login_attempt v2" 2024-09-11T08:08:11.200 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.747290831Z level=info msg="Executing migration" id="create index IDX_login_attempt_username - v2" 2024-09-11T08:08:11.200 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.747700343Z level=info msg="Executing migration" id="copy login_attempt v1 to v2" 2024-09-11T08:08:11.200 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.747983335Z level=info msg="Executing migration" id="drop login_attempt_tmp_qwerty" 2024-09-11T08:08:11.200 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.748350316Z level=info msg="Executing migration" id="create user auth table" 2024-09-11T08:08:11.200 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.74869327Z level=info msg="Executing migration" id="create index IDX_user_auth_auth_module_auth_id - v1" 2024-09-11T08:08:11.200 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.749122686Z level=info msg="Executing migration" id="alter user_auth.auth_id to length 190" 2024-09-11T08:08:11.200 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.749415269Z level=info msg="Executing migration" id="Add OAuth access token to user_auth" 2024-09-11T08:08:11.200 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.750107622Z level=info msg="Executing migration" id="Add OAuth refresh token to user_auth" 2024-09-11T08:08:11.201 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.750821587Z level=info msg="Executing migration" id="Add OAuth token type to user_auth" 2024-09-11T08:08:11.201 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.751542465Z level=info msg="Executing migration" id="Add OAuth expiry to user_auth" 2024-09-11T08:08:11.201 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.752302737Z level=info msg="Executing migration" id="Add index to user_id column in user_auth" 2024-09-11T08:08:11.201 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.752711551Z level=info msg="Executing migration" id="Add OAuth ID token to user_auth" 2024-09-11T08:08:11.201 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.753435884Z level=info msg="Executing migration" id="create server_lock table" 2024-09-11T08:08:11.201 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.753816883Z level=info msg="Executing migration" id="add index server_lock.operation_uid" 2024-09-11T08:08:11.201 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.754252764Z level=info msg="Executing migration" id="create user auth token table" 2024-09-11T08:08:11.201 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.754678671Z level=info msg="Executing migration" id="add unique index user_auth_token.auth_token" 2024-09-11T08:08:11.201 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.757448479Z level=info msg="Executing migration" id="add unique index user_auth_token.prev_auth_token" 2024-09-11T08:08:11.201 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.757885147Z level=info msg="Executing migration" id="add index user_auth_token.user_id" 2024-09-11T08:08:11.201 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.758336487Z level=info msg="Executing migration" id="Add revoked_at to the user auth token" 2024-09-11T08:08:11.201 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.759084167Z level=info msg="Executing migration" id="create cache_data table" 2024-09-11T08:08:11.202 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.759502308Z level=info msg="Executing migration" id="add unique index cache_data.cache_key" 2024-09-11T08:08:11.202 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.759936401Z level=info msg="Executing migration" id="create short_url table v1" 2024-09-11T08:08:11.202 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.760382537Z level=info msg="Executing migration" id="add index short_url.org_id-uid" 2024-09-11T08:08:11.202 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.763195854Z level=info msg="Executing migration" id="alter table short_url alter column created_by type to bigint" 2024-09-11T08:08:11.202 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.763645864Z level=info msg="Executing migration" id="delete alert_definition table" 2024-09-11T08:08:11.202 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.763944217Z level=info msg="Executing migration" id="recreate alert_definition table" 2024-09-11T08:08:11.202 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.764400963Z level=info msg="Executing migration" id="add index in alert_definition on org_id and title columns" 2024-09-11T08:08:11.202 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.767197481Z level=info msg="Executing migration" id="add index in alert_definition on org_id and uid columns" 2024-09-11T08:08:11.202 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.767729414Z level=info msg="Executing migration" id="alter alert_definition table data column to mediumtext in mysql" 2024-09-11T08:08:11.202 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.768026916Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and title columns" 2024-09-11T08:08:11.202 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.768524669Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and uid columns" 2024-09-11T08:08:11.202 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.768974657Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and title columns" 2024-09-11T08:08:11.203 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.769494301Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and uid columns" 2024-09-11T08:08:11.203 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.769959937Z level=info msg="Executing migration" id="Add column paused in alert_definition" 2024-09-11T08:08:11.203 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.770777953Z level=info msg="Executing migration" id="drop alert_definition table" 2024-09-11T08:08:11.203 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.771224902Z level=info msg="Executing migration" id="delete alert_definition_version table" 2024-09-11T08:08:11.203 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.771538281Z level=info msg="Executing migration" id="recreate alert_definition_version table" 2024-09-11T08:08:11.203 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.771950571Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_id and version columns" 2024-09-11T08:08:11.203 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.772417278Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_uid and version columns" 2024-09-11T08:08:11.203 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.773055707Z level=info msg="Executing migration" id="alter alert_definition_version table data column to mediumtext in mysql" 2024-09-11T08:08:11.203 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.773371942Z level=info msg="Executing migration" id="drop alert_definition_version table" 2024-09-11T08:08:11.203 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.773795272Z level=info msg="Executing migration" id="create alert_instance table" 2024-09-11T08:08:11.203 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.774238171Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, def_uid and current_state columns" 2024-09-11T08:08:11.203 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.774711555Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, current_state columns" 2024-09-11T08:08:11.204 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.775201604Z level=info msg="Executing migration" id="add column current_state_end to alert_instance" 2024-09-11T08:08:11.204 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.776041896Z level=info msg="Executing migration" id="remove index def_org_id, def_uid, current_state on alert_instance" 2024-09-11T08:08:11.204 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.776478882Z level=info msg="Executing migration" id="remove index def_org_id, current_state on alert_instance" 2024-09-11T08:08:11.204 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.776916909Z level=info msg="Executing migration" id="rename def_org_id to rule_org_id in alert_instance" 2024-09-11T08:08:11.204 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.780319662Z level=info msg="Executing migration" id="rename def_uid to rule_uid in alert_instance" 2024-09-11T08:08:11.204 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.783650997Z level=info msg="Executing migration" id="add index rule_org_id, rule_uid, current_state on alert_instance" 2024-09-11T08:08:11.204 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.784115378Z level=info msg="Executing migration" id="add index rule_org_id, current_state on alert_instance" 2024-09-11T08:08:11.204 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.786698492Z level=info msg="Executing migration" id="add current_reason column related to current_state" 2024-09-11T08:08:11.204 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.787562958Z level=info msg="Executing migration" id="create alert_rule table" 2024-09-11T08:08:11.204 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.788037218Z level=info msg="Executing migration" id="add index in alert_rule on org_id and title columns" 2024-09-11T08:08:11.204 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.790641259Z level=info msg="Executing migration" id="add index in alert_rule on org_id and uid columns" 2024-09-11T08:08:11.204 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.793373487Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespace_uid, group_uid columns" 2024-09-11T08:08:11.205 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.795945931Z level=info msg="Executing migration" id="alter alert_rule table data column to mediumtext in mysql" 2024-09-11T08:08:11.205 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.798669612Z level=info msg="Executing migration" id="add column for to alert_rule" 2024-09-11T08:08:11.205 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.801486075Z level=info msg="Executing migration" id="add column annotations to alert_rule" 2024-09-11T08:08:11.205 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.804303307Z level=info msg="Executing migration" id="add column labels to alert_rule" 2024-09-11T08:08:11.205 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.807036604Z level=info msg="Executing migration" id="remove unique index from alert_rule on org_id, title columns" 2024-09-11T08:08:11.205 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.809990742Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespase_uid and title columns" 2024-09-11T08:08:11.205 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.812443547Z level=info msg="Executing migration" id="add dashboard_uid column to alert_rule" 2024-09-11T08:08:11.205 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.815081851Z level=info msg="Executing migration" id="add panel_id column to alert_rule" 2024-09-11T08:08:11.205 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.81747951Z level=info msg="Executing migration" id="add index in alert_rule on org_id, dashboard_uid and panel_id columns" 2024-09-11T08:08:11.205 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.820058639Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule" 2024-09-11T08:08:11.205 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.822497655Z level=info msg="Executing migration" id="add is_paused column to alert_rule table" 2024-09-11T08:08:11.205 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.825072666Z level=info msg="Executing migration" id="fix is_paused column for alert_rule table" 2024-09-11T08:08:11.206 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.827709404Z level=info msg="Executing migration" id="create alert_rule_version table" 2024-09-11T08:08:11.206 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.830159539Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_uid and version columns" 2024-09-11T08:08:11.206 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.832610867Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_namespace_uid and rule_group columns" 2024-09-11T08:08:11.206 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.835207178Z level=info msg="Executing migration" id="alter alert_rule_version table data column to mediumtext in mysql" 2024-09-11T08:08:11.206 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.835546748Z level=info msg="Executing migration" id="add column for to alert_rule_version" 2024-09-11T08:08:11.206 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.836415019Z level=info msg="Executing migration" id="add column annotations to alert_rule_version" 2024-09-11T08:08:11.206 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.837262882Z level=info msg="Executing migration" id="add column labels to alert_rule_version" 2024-09-11T08:08:11.206 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.838140237Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule_version" 2024-09-11T08:08:11.206 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.839030559Z level=info msg="Executing migration" id="add is_paused column to alert_rule_versions table" 2024-09-11T08:08:11.206 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.839874046Z level=info msg="Executing migration" id="fix is_paused column for alert_rule_version table" 2024-09-11T08:08:11.206 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.840654925Z level=info msg="Executing migration" id=create_alert_configuration_table 2024-09-11T08:08:11.206 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.841522202Z level=info msg="Executing migration" id="Add column default in alert_configuration" 2024-09-11T08:08:11.207 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.842386096Z level=info msg="Executing migration" id="alert alert_configuration alertmanager_configuration column from TEXT to MEDIUMTEXT if mysql" 2024-09-11T08:08:11.207 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.843267946Z level=info msg="Executing migration" id="add column org_id in alert_configuration" 2024-09-11T08:08:11.207 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.844156095Z level=info msg="Executing migration" id="add index in alert_configuration table on org_id column" 2024-09-11T08:08:11.207 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.844940303Z level=info msg="Executing migration" id="add configuration_hash column to alert_configuration" 2024-09-11T08:08:11.207 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.845793694Z level=info msg="Executing migration" id=create_ngalert_configuration_table 2024-09-11T08:08:11.207 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.846638017Z level=info msg="Executing migration" id="add index in ngalert_configuration on org_id column" 2024-09-11T08:08:11.207 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.847641009Z level=info msg="Executing migration" id="add column send_alerts_to in ngalert_configuration" 2024-09-11T08:08:11.207 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.848549832Z level=info msg="Executing migration" id="create provenance_type table" 2024-09-11T08:08:11.207 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.849329157Z level=info msg="Executing migration" id="add index to uniquify (record_key, record_type, org_id) columns" 2024-09-11T08:08:11.207 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.850198546Z level=info msg="Executing migration" id="create alert_image table" 2024-09-11T08:08:11.207 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.85108691Z level=info msg="Executing migration" id="add unique index on token to alert_image table" 2024-09-11T08:08:11.207 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.853908108Z level=info msg="Executing migration" id="support longer URLs in alert_image table" 2024-09-11T08:08:11.208 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.856670912Z level=info msg="Executing migration" id=create_alert_configuration_history_table 2024-09-11T08:08:11.208 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.859530463Z level=info msg="Executing migration" id="drop non-unique orgID index on alert_configuration" 2024-09-11T08:08:11.208 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.862489541Z level=info msg="Executing migration" id="drop unique orgID index on alert_configuration if exists" 2024-09-11T08:08:11.208 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.862699148Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop unique orgID index on alert_configuration if exists" 2024-09-11T08:08:11.208 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.865146061Z level=info msg="Executing migration" id="extract alertmanager configuration history to separate table" 2024-09-11T08:08:11.208 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.867820182Z level=info msg="Executing migration" id="add unique index on orgID to alert_configuration" 2024-09-11T08:08:11.208 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.870634339Z level=info msg="Executing migration" id="clear migration entry \"remove unified alerting data\"" 2024-09-11T08:08:11.208 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.870766141Z level=info msg="Executing migration" id="move dashboard alerts to unified alerting" 2024-09-11T08:08:11.208 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.87087928Z level=info msg="alerts found to migrate" alerts=0 2024-09-11T08:08:11.208 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.873675949Z level=info msg="Executing migration" id="create library_element table v1" 2024-09-11T08:08:11.208 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.876487003Z level=info msg="Executing migration" id="add index library_element org_id-folder_id-name-kind" 2024-09-11T08:08:11.209 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.879023289Z level=info msg="Executing migration" id="create library_element_connection table v1" 2024-09-11T08:08:11.209 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.881860162Z level=info msg="Executing migration" id="add index library_element_connection element_id-kind-connection_id" 2024-09-11T08:08:11.209 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.884568095Z level=info msg="Executing migration" id="add unique index library_element org_id_uid" 2024-09-11T08:08:11.209 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.887349942Z level=info msg="Executing migration" id="increase max description length to 2048" 2024-09-11T08:08:11.209 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.890174426Z level=info msg="Executing migration" id="clone move dashboard alerts to unified alerting" 2024-09-11T08:08:11.209 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.892708915Z level=info msg="Executing migration" id="create data_keys table" 2024-09-11T08:08:11.209 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.89565648Z level=info msg="Executing migration" id="create secrets table" 2024-09-11T08:08:11.209 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.898446467Z level=info msg="Executing migration" id="rename data_keys name column to id" 2024-09-11T08:08:11.209 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.902735748Z level=info msg="Executing migration" id="add name column into data_keys" 2024-09-11T08:08:11.209 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.905166704Z level=info msg="Executing migration" id="copy data_keys id column values into name" 2024-09-11T08:08:11.209 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.907879647Z level=info msg="Executing migration" id="rename data_keys name column to label" 2024-09-11T08:08:11.209 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.916172396Z level=info msg="Executing migration" id="rename data_keys id column back to name" 2024-09-11T08:08:11.210 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.920397789Z level=info msg="Executing migration" id="create kv_store table v1" 2024-09-11T08:08:11.210 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.923005647Z level=info msg="Executing migration" id="add index kv_store.org_id-namespace-key" 2024-09-11T08:08:11.210 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.925368185Z level=info msg="Executing migration" id="update dashboard_uid and panel_id from existing annotations" 2024-09-11T08:08:11.210 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.928054863Z level=info msg="Executing migration" id="create permission table" 2024-09-11T08:08:11.210 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.930744832Z level=info msg="Executing migration" id="add unique index permission.role_id" 2024-09-11T08:08:11.210 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.933215468Z level=info msg="Executing migration" id="add unique index role_id_action_scope" 2024-09-11T08:08:11.210 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.93544658Z level=info msg="Executing migration" id="create role table" 2024-09-11T08:08:11.210 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.938177095Z level=info msg="Executing migration" id="add column display_name" 2024-09-11T08:08:11.210 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.940968465Z level=info msg="Executing migration" id="add column group_name" 2024-09-11T08:08:11.210 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.943710982Z level=info msg="Executing migration" id="add index role.org_id" 2024-09-11T08:08:11.210 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.946171386Z level=info msg="Executing migration" id="add unique index role_org_id_name" 2024-09-11T08:08:11.211 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.949055697Z level=info msg="Executing migration" id="add index role_org_id_uid" 2024-09-11T08:08:11.211 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.9516043Z level=info msg="Executing migration" id="create team role table" 2024-09-11T08:08:11.211 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.954352669Z level=info msg="Executing migration" id="add index team_role.org_id" 2024-09-11T08:08:11.211 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.955074938Z level=info msg="Executing migration" id="add unique index team_role_org_id_team_id_role_id" 2024-09-11T08:08:11.211 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.956086005Z level=info msg="Executing migration" id="add index team_role.team_id" 2024-09-11T08:08:11.211 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.956781457Z level=info msg="Executing migration" id="create user role table" 2024-09-11T08:08:11.211 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.957649193Z level=info msg="Executing migration" id="add index user_role.org_id" 2024-09-11T08:08:11.211 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.958430089Z level=info msg="Executing migration" id="add unique index user_role_org_id_user_id_role_id" 2024-09-11T08:08:11.211 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.959215895Z level=info msg="Executing migration" id="add index user_role.user_id" 2024-09-11T08:08:11.211 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.960079469Z level=info msg="Executing migration" id="create builtin role table" 2024-09-11T08:08:11.211 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.960926003Z level=info msg="Executing migration" id="add index builtin_role.role_id" 2024-09-11T08:08:11.211 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.961811502Z level=info msg="Executing migration" id="add index builtin_role.name" 2024-09-11T08:08:11.211 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.962599715Z level=info msg="Executing migration" id="Add column org_id to builtin_role table" 2024-09-11T08:08:11.212 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.963609459Z level=info msg="Executing migration" id="add index builtin_role.org_id" 2024-09-11T08:08:11.212 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.964450551Z level=info msg="Executing migration" id="add unique index builtin_role_org_id_role_id_role" 2024-09-11T08:08:11.212 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.965329814Z level=info msg="Executing migration" id="Remove unique index role_org_id_uid" 2024-09-11T08:08:11.212 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.966291825Z level=info msg="Executing migration" id="add unique index role.uid" 2024-09-11T08:08:11.212 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.966912555Z level=info msg="Executing migration" id="create seed assignment table" 2024-09-11T08:08:11.212 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.967903167Z level=info msg="Executing migration" id="add unique index builtin_role_role_name" 2024-09-11T08:08:11.212 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.968651889Z level=info msg="Executing migration" id="add column hidden to role table" 2024-09-11T08:08:11.212 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.969657557Z level=info msg="Executing migration" id="create query_history table v1" 2024-09-11T08:08:11.212 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.97043711Z level=info msg="Executing migration" id="add index query_history.org_id-created_by-datasource_uid" 2024-09-11T08:08:11.212 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.971320402Z level=info msg="Executing migration" id="alter table query_history alter column created_by type to bigint" 2024-09-11T08:08:11.212 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.972194591Z level=info msg="Executing migration" id="teams permissions migration" 2024-09-11T08:08:11.228 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.973086291Z level=info msg="Executing migration" id="dashboard permissions" 2024-09-11T08:08:11.228 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.974042325Z level=info msg="Executing migration" id="dashboard permissions uid scopes" 2024-09-11T08:08:11.228 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.974801867Z level=info msg="Executing migration" id="drop managed folder create actions" 2024-09-11T08:08:11.228 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.975657963Z level=info msg="Executing migration" id="alerting notification permissions" 2024-09-11T08:08:11.228 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.976032308Z level=info msg="Executing migration" id="create query_history_star table v1" 2024-09-11T08:08:11.228 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.976575062Z level=info msg="Executing migration" id="add index query_history.user_id-query_uid" 2024-09-11T08:08:11.229 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.977435725Z level=info msg="Executing migration" id="add column org_id in query_history_star" 2024-09-11T08:08:11.229 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.978410002Z level=info msg="Executing migration" id="alter table query_history_star_mig column user_id type to bigint" 2024-09-11T08:08:11.229 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.979302378Z level=info msg="Executing migration" id="create correlation table v1" 2024-09-11T08:08:11.229 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.98017501Z level=info msg="Executing migration" id="add index correlations.uid" 2024-09-11T08:08:11.229 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.981192363Z level=info msg="Executing migration" id="add index correlations.source_uid" 2024-09-11T08:08:11.229 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.981939645Z level=info msg="Executing migration" id="add correlation config column" 2024-09-11T08:08:11.229 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.982935595Z level=info msg="Executing migration" id="create entity_events table" 2024-09-11T08:08:11.229 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.983833099Z level=info msg="Executing migration" id="create dashboard public config v1" 2024-09-11T08:08:11.229 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.986507957Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v1" 2024-09-11T08:08:11.229 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.986600223Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index UQE_dashboard_public_config_uid - v1" 2024-09-11T08:08:11.229 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.989435923Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-11T08:08:11.230 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.989542717Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-11T08:08:11.230 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.99224066Z level=info msg="Executing migration" id="Drop old dashboard public config table" 2024-09-11T08:08:11.230 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.995208219Z level=info msg="Executing migration" id="recreate dashboard public config v1" 2024-09-11T08:08:11.230 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:10.998044507Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v1" 2024-09-11T08:08:11.231 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:11 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:11.000521796Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-11T08:08:11.231 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:11 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:11.003479831Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v2" 2024-09-11T08:08:11.231 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:11 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:11.006388659Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-09-11T08:08:11.231 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:11 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:11.009190325Z level=info msg="Executing migration" id="Drop public config table" 2024-09-11T08:08:11.231 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:11 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:11.011884812Z level=info msg="Executing migration" id="Recreate dashboard public config v2" 2024-09-11T08:08:11.231 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:11 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:11.014744647Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v2" 2024-09-11T08:08:11.231 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:11 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:11.017543419Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-09-11T08:08:11.231 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:11 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:11.020432265Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_access_token - v2" 2024-09-11T08:08:11.231 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:11 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:11.023061188Z level=info msg="Executing migration" id="Rename table dashboard_public_config to dashboard_public - v2" 2024-09-11T08:08:11.231 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:11 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:11.026240286Z level=info msg="Executing migration" id="add annotations_enabled column" 2024-09-11T08:08:11.231 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:11 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:11.029151059Z level=info msg="Executing migration" id="add time_selection_enabled column" 2024-09-11T08:08:11.231 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:11 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:11.031847796Z level=info msg="Executing migration" id="delete orphaned public dashboards" 2024-09-11T08:08:11.232 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:11 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:11.03455616Z level=info msg="Executing migration" id="add share column" 2024-09-11T08:08:11.232 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:11 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:11.037230746Z level=info msg="Executing migration" id="create default alerting folders" 2024-09-11T08:08:11.232 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:11 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:11.039951577Z level=info msg="Executing migration" id="create file table" 2024-09-11T08:08:11.232 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:11 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:11.042471444Z level=info msg="Executing migration" id="file table idx: path natural pk" 2024-09-11T08:08:11.232 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:11 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:11.04516457Z level=info msg="Executing migration" id="file table idx: parent_folder_path_hash fast folder retrieval" 2024-09-11T08:08:11.232 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:11 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:11.047690738Z level=info msg="Executing migration" id="create file_meta table" 2024-09-11T08:08:11.232 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:11 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:11.050343986Z level=info msg="Executing migration" id="file table idx: path key" 2024-09-11T08:08:11.232 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:11 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:11.052995268Z level=info msg="Executing migration" id="set path collation in file table" 2024-09-11T08:08:11.232 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:11 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:11.055876632Z level=info msg="Executing migration" id="managed permissions migration" 2024-09-11T08:08:11.232 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:11 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:11.058668323Z level=info msg="Executing migration" id="managed folder permissions alert actions migration" 2024-09-11T08:08:11.232 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:11 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:11.061037627Z level=info msg="Executing migration" id="RBAC action name migrator" 2024-09-11T08:08:11.232 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:11 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:11.063696284Z level=info msg="Executing migration" id="Add UID column to playlist" 2024-09-11T08:08:11.232 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:11 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:11.066503445Z level=info msg="Executing migration" id="Update uid column values in playlist" 2024-09-11T08:08:11.233 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:11 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:11.069226752Z level=info msg="Executing migration" id="Add index for uid in playlist" 2024-09-11T08:08:11.233 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:11 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:11.071736893Z level=info msg="Executing migration" id="update group index for alert rules" 2024-09-11T08:08:11.233 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:11 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:11.074279262Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated migration" 2024-09-11T08:08:11.233 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:11 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:11.077049975Z level=info msg="Executing migration" id="admin only folder/dashboard permission" 2024-09-11T08:08:11.233 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:11 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:11.079723338Z level=info msg="Executing migration" id="add action column to seed_assignment" 2024-09-11T08:08:11.233 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:11 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:11.080773602Z level=info msg="Executing migration" id="add scope column to seed_assignment" 2024-09-11T08:08:11.233 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:11 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:11.081808815Z level=info msg="Executing migration" id="remove unique index builtin_role_role_name before nullable update" 2024-09-11T08:08:11.233 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:11 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:11.082679255Z level=info msg="Executing migration" id="update seed_assignment role_name column to nullable" 2024-09-11T08:08:11.233 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:11 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:11.092309359Z level=info msg="Executing migration" id="add unique index builtin_role_name back" 2024-09-11T08:08:11.233 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:11 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:11.093113153Z level=info msg="Executing migration" id="add unique index builtin_role_action_scope" 2024-09-11T08:08:11.233 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:11 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:11.09385896Z level=info msg="Executing migration" id="add primary key to seed_assigment" 2024-09-11T08:08:11.233 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:11 smithi135 bash[63259]: 3d910835e226b370d305848077dbaa374ad787eb07a96ff345b4ee31f93aeaaa 2024-09-11T08:08:11.234 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:11 smithi135 systemd[1]: Started Ceph grafana.a for f1d75850-7013-11ef-bcea-c7b262605968. 2024-09-11T08:08:11.234 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:11 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:11.098577621Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated fixed migration" 2024-09-11T08:08:11.234 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:11 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:11.099066259Z level=info msg="Executing migration" id="migrate external alertmanagers to datsourcse" 2024-09-11T08:08:11.234 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:11 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:11.099935128Z level=info msg="Executing migration" id="create folder table" 2024-09-11T08:08:11.234 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:11 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:11.100755249Z level=info msg="Executing migration" id="Add index for parent_uid" 2024-09-11T08:08:11.234 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:11 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:11.101593023Z level=info msg="Executing migration" id="Add unique index for folder.uid and folder.org_id" 2024-09-11T08:08:11.234 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:11 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:11.106985113Z level=info msg="Executing migration" id="Update folder title length" 2024-09-11T08:08:11.234 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:11 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:11.107382937Z level=info msg="Executing migration" id="Add unique index for folder.title and folder.parent_uid" 2024-09-11T08:08:11.234 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:11 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=migrator t=2024-09-11T08:08:11.108232448Z level=info msg="migrations completed" performed=485 skipped=0 duration=728.624997ms 2024-09-11T08:08:11.234 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:11 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=sqlstore t=2024-09-11T08:08:11.108656378Z level=info msg="Created default organization" 2024-09-11T08:08:11.234 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:11 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=plugin.loader t=2024-09-11T08:08:11.168468347Z level=info msg="Plugin registered" pluginID=input 2024-09-11T08:08:11.234 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:11 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=plugin.loader t=2024-09-11T08:08:11.198711676Z level=info msg="Plugin registered" pluginID=grafana-piechart-panel 2024-09-11T08:08:11.234 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:11 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=plugin.loader t=2024-09-11T08:08:11.198739933Z level=info msg="Plugin registered" pluginID=vonage-status-panel 2024-09-11T08:08:11.235 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:11 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=secrets t=2024-09-11T08:08:11.198781282Z level=info msg="Envelope encryption state" enabled=true currentprovider=secretKey.v1 2024-09-11T08:08:11.235 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:11 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=query_data t=2024-09-11T08:08:11.200595387Z level=info msg="Query Service initialization" 2024-09-11T08:08:11.235 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:11 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=live.push_http t=2024-09-11T08:08:11.202945369Z level=info msg="Live Push Gateway initialization" 2024-09-11T08:08:11.459 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:08:12.228 INFO:journalctl@ceph.iscsi.iscsi.a.smithi135.stdout:Sep 11 08:08:11 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[59799]: debug there is no tcmu-runner data available 2024-09-11T08:08:12.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:12 smithi135 ceph-mon[32689]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:08:12.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:12 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:12.229 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:12 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:12.229 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:12 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:12.229 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:12 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:12.229 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:12 smithi135 ceph-mon[32689]: Deploying daemon node-exporter.a on smithi122 2024-09-11T08:08:12.229 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:12 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:12.229 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:11 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=infra.usagestats.collector t=2024-09-11T08:08:11.924266542Z level=info msg="registering usage stat providers" usageStatsProvidersLen=2 2024-09-11T08:08:12.229 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:11 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=provisioning.datasources t=2024-09-11T08:08:11.924670154Z level=warn msg="[Deprecated] the datasource provisioning config is outdated. please upgrade" filename=/etc/grafana/provisioning/datasources/ceph-dashboard.yml 2024-09-11T08:08:12.229 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:11 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=provisioning.datasources t=2024-09-11T08:08:11.958414973Z level=info msg="inserting datasource from configuration " name=Dashboard1 uid=P43CA22E17D0F9596 2024-09-11T08:08:12.229 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:11 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=provisioning.datasources t=2024-09-11T08:08:11.990135724Z level=info msg="inserting datasource from configuration " name=Loki uid=P8E80F9AEF21F6940 2024-09-11T08:08:12.229 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:12 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=provisioning.alerting t=2024-09-11T08:08:12.0115628Z level=info msg="starting to provision alerting" 2024-09-11T08:08:12.229 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:12 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=provisioning.alerting t=2024-09-11T08:08:12.011588115Z level=info msg="finished to provision alerting" 2024-09-11T08:08:12.230 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:12 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=grafanaStorageLogger t=2024-09-11T08:08:12.011756337Z level=info msg="storage starting" 2024-09-11T08:08:12.230 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:12 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=ngalert.state.manager t=2024-09-11T08:08:12.011858966Z level=info msg="Warming state cache for startup" 2024-09-11T08:08:12.230 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:12 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=http.server t=2024-09-11T08:08:12.013510854Z level=info msg="HTTP Server Listen" address=[::]:3000 protocol=https subUrl= socket= 2024-09-11T08:08:12.230 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:12 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=sqlstore.transactions t=2024-09-11T08:08:12.022884987Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-11T08:08:12.230 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:12 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=ngalert.state.manager t=2024-09-11T08:08:12.022940958Z level=info msg="State cache has been initialized" states=0 duration=11.080957ms 2024-09-11T08:08:12.230 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:12 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=ticker t=2024-09-11T08:08:12.023012873Z level=info msg=starting first_tick=2024-09-11T08:08:20Z 2024-09-11T08:08:12.230 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:12 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=ngalert.multiorg.alertmanager t=2024-09-11T08:08:12.023009236Z level=info msg="starting MultiOrg Alertmanager" 2024-09-11T08:08:12.230 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:12 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=sqlstore.transactions t=2024-09-11T08:08:12.06924836Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-11T08:08:12.230 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:08:12 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a[63273]: logger=sqlstore.transactions t=2024-09-11T08:08:12.104123027Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-11T08:08:12.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:12 smithi122 ceph-mon[28992]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:08:12.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:12 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:12.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:12 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:12.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:12 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:12.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:12 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:12.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:12 smithi122 ceph-mon[28992]: Deploying daemon node-exporter.a on smithi122 2024-09-11T08:08:12.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:12 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:12.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:12 smithi122 ceph-mon[39726]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:08:12.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:12 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:12.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:12 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:12.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:12 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:12.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:12 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:12.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:12 smithi122 ceph-mon[39726]: Deploying daemon node-exporter.a on smithi122 2024-09-11T08:08:12.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:12 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:13.283 INFO:teuthology.orchestra.run.smithi122.stdout:rotating key for osd.0 2024-09-11T08:08:13.284 INFO:teuthology.orchestra.run.smithi122.stderr:+ for f in osd.0 osd.1 osd.2 osd.3 osd.4 osd.5 osd.6 osd.7 mgr.y mgr.x 2024-09-11T08:08:13.284 INFO:teuthology.orchestra.run.smithi122.stderr:+ echo 'rotating key for osd.0' 2024-09-11T08:08:13.284 INFO:teuthology.orchestra.run.smithi122.stderr:++ ceph auth get-key osd.0 2024-09-11T08:08:13.454 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:13 smithi122 ceph-mon[39726]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:08:13.455 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:13 smithi122 ceph-mon[28992]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:08:13.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:13 smithi135 ceph-mon[32689]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:08:13.529 INFO:teuthology.orchestra.run.smithi122.stderr:+ K=AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== 2024-09-11T08:08:13.529 INFO:teuthology.orchestra.run.smithi122.stderr:+ NK=AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== 2024-09-11T08:08:13.529 INFO:teuthology.orchestra.run.smithi122.stderr:+ ceph orch daemon rotate-key osd.0 2024-09-11T08:08:13.787 INFO:teuthology.orchestra.run.smithi122.stdout:Scheduled to rotate-key osd.0 on host 'smithi122' 2024-09-11T08:08:13.798 INFO:teuthology.orchestra.run.smithi122.stderr:+ '[' AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== == AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== ']' 2024-09-11T08:08:13.798 INFO:teuthology.orchestra.run.smithi122.stderr:+ sleep 5 2024-09-11T08:08:14.170 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Sep 11 08:08:13 smithi122 bash[72771]: Copying blob sha256:5c12815fee558b157dee7f7509dedbaba0a8379098858a65ec869e1f1526ea0c 2024-09-11T08:08:14.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:14 smithi135 ceph-mon[32689]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:08:14.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:14 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/2777750275' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:08:14.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:14 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:14.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:14 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:14.483 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:14 smithi122 ceph-mon[28992]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:08:14.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:14 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/2777750275' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:08:14.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:14 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:14.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:14 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:14.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:14 smithi122 ceph-mon[39726]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:08:14.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:14 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/2777750275' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:08:14.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:14 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:14.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:14 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:15.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:15 smithi135 ceph-mon[32689]: from='client.24605 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:08:15.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:15 smithi135 ceph-mon[32689]: Schedule rotate-key daemon osd.0 2024-09-11T08:08:15.483 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:15 smithi122 ceph-mon[28992]: from='client.24605 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:08:15.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:15 smithi122 ceph-mon[28992]: Schedule rotate-key daemon osd.0 2024-09-11T08:08:15.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:15 smithi122 ceph-mon[39726]: from='client.24605 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:08:15.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:15 smithi122 ceph-mon[39726]: Schedule rotate-key daemon osd.0 2024-09-11T08:08:15.484 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Sep 11 08:08:15 smithi122 bash[72771]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-09-11T08:08:15.484 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Sep 11 08:08:15 smithi122 bash[72771]: Writing manifest to image destination 2024-09-11T08:08:15.874 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Sep 11 08:08:15 smithi122 podman[72771]: 2024-09-11 08:08:15.548133129 +0000 UTC m=+2.207641978 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-09-11T08:08:15.874 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Sep 11 08:08:15 smithi122 podman[72771]: 2024-09-11 08:08:15.671772984 +0000 UTC m=+2.331281832 container create f25b60c5acc6bffcf0683390d9d9726c724df1582a7c4d3fc840425271e6db01 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-11T08:08:16.178 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Sep 11 08:08:15 smithi122 podman[72771]: 2024-09-11 08:08:15.873880885 +0000 UTC m=+2.533389734 container init f25b60c5acc6bffcf0683390d9d9726c724df1582a7c4d3fc840425271e6db01 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-11T08:08:16.179 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Sep 11 08:08:15 smithi122 podman[72771]: 2024-09-11 08:08:15.877149938 +0000 UTC m=+2.536658803 container start f25b60c5acc6bffcf0683390d9d9726c724df1582a7c4d3fc840425271e6db01 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-11T08:08:16.179 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Sep 11 08:08:15 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-a[72871]: ts=2024-09-11T08:08:15.881Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-09-11T08:08:16.179 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Sep 11 08:08:15 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-a[72871]: ts=2024-09-11T08:08:15.881Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-09-11T08:08:16.179 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Sep 11 08:08:15 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-a[72871]: ts=2024-09-11T08:08:15.881Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-09-11T08:08:16.179 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Sep 11 08:08:15 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-a[72871]: ts=2024-09-11T08:08:15.881Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-09-11T08:08:16.179 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Sep 11 08:08:15 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-a[72871]: ts=2024-09-11T08:08:15.882Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-09-11T08:08:16.179 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Sep 11 08:08:15 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-a[72871]: ts=2024-09-11T08:08:15.882Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-09-11T08:08:16.179 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Sep 11 08:08:15 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-a[72871]: ts=2024-09-11T08:08:15.882Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-09-11T08:08:16.179 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Sep 11 08:08:15 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-a[72871]: ts=2024-09-11T08:08:15.882Z caller=node_exporter.go:117 level=info collector=arp 2024-09-11T08:08:16.179 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Sep 11 08:08:15 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-a[72871]: ts=2024-09-11T08:08:15.882Z caller=node_exporter.go:117 level=info collector=bcache 2024-09-11T08:08:16.179 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Sep 11 08:08:15 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-a[72871]: ts=2024-09-11T08:08:15.882Z caller=node_exporter.go:117 level=info collector=bonding 2024-09-11T08:08:16.180 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Sep 11 08:08:15 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-a[72871]: ts=2024-09-11T08:08:15.882Z caller=node_exporter.go:117 level=info collector=btrfs 2024-09-11T08:08:16.180 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Sep 11 08:08:15 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-a[72871]: ts=2024-09-11T08:08:15.882Z caller=node_exporter.go:117 level=info collector=conntrack 2024-09-11T08:08:16.180 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Sep 11 08:08:15 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-a[72871]: ts=2024-09-11T08:08:15.882Z caller=node_exporter.go:117 level=info collector=cpu 2024-09-11T08:08:16.180 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Sep 11 08:08:15 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-a[72871]: ts=2024-09-11T08:08:15.882Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-09-11T08:08:16.180 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Sep 11 08:08:15 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-a[72871]: ts=2024-09-11T08:08:15.882Z caller=node_exporter.go:117 level=info collector=diskstats 2024-09-11T08:08:16.180 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Sep 11 08:08:15 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-a[72871]: ts=2024-09-11T08:08:15.882Z caller=node_exporter.go:117 level=info collector=dmi 2024-09-11T08:08:16.180 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Sep 11 08:08:15 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-a[72871]: ts=2024-09-11T08:08:15.882Z caller=node_exporter.go:117 level=info collector=edac 2024-09-11T08:08:16.181 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Sep 11 08:08:15 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-a[72871]: ts=2024-09-11T08:08:15.882Z caller=node_exporter.go:117 level=info collector=entropy 2024-09-11T08:08:16.181 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Sep 11 08:08:15 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-a[72871]: ts=2024-09-11T08:08:15.882Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-09-11T08:08:16.181 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Sep 11 08:08:15 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-a[72871]: ts=2024-09-11T08:08:15.882Z caller=node_exporter.go:117 level=info collector=filefd 2024-09-11T08:08:16.181 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Sep 11 08:08:15 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-a[72871]: ts=2024-09-11T08:08:15.882Z caller=node_exporter.go:117 level=info collector=filesystem 2024-09-11T08:08:16.181 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Sep 11 08:08:15 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-a[72871]: ts=2024-09-11T08:08:15.882Z caller=node_exporter.go:117 level=info collector=hwmon 2024-09-11T08:08:16.181 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Sep 11 08:08:15 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-a[72871]: ts=2024-09-11T08:08:15.882Z caller=node_exporter.go:117 level=info collector=infiniband 2024-09-11T08:08:16.181 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Sep 11 08:08:15 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-a[72871]: ts=2024-09-11T08:08:15.882Z caller=node_exporter.go:117 level=info collector=ipvs 2024-09-11T08:08:16.181 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Sep 11 08:08:15 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-a[72871]: ts=2024-09-11T08:08:15.882Z caller=node_exporter.go:117 level=info collector=loadavg 2024-09-11T08:08:16.181 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Sep 11 08:08:15 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-a[72871]: ts=2024-09-11T08:08:15.882Z caller=node_exporter.go:117 level=info collector=mdadm 2024-09-11T08:08:16.181 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Sep 11 08:08:15 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-a[72871]: ts=2024-09-11T08:08:15.882Z caller=node_exporter.go:117 level=info collector=meminfo 2024-09-11T08:08:16.181 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Sep 11 08:08:15 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-a[72871]: ts=2024-09-11T08:08:15.882Z caller=node_exporter.go:117 level=info collector=netclass 2024-09-11T08:08:16.181 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Sep 11 08:08:15 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-a[72871]: ts=2024-09-11T08:08:15.882Z caller=node_exporter.go:117 level=info collector=netdev 2024-09-11T08:08:16.182 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Sep 11 08:08:15 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-a[72871]: ts=2024-09-11T08:08:15.882Z caller=node_exporter.go:117 level=info collector=netstat 2024-09-11T08:08:16.182 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Sep 11 08:08:15 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-a[72871]: ts=2024-09-11T08:08:15.882Z caller=node_exporter.go:117 level=info collector=nfs 2024-09-11T08:08:16.182 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Sep 11 08:08:15 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-a[72871]: ts=2024-09-11T08:08:15.882Z caller=node_exporter.go:117 level=info collector=nfsd 2024-09-11T08:08:16.182 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Sep 11 08:08:15 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-a[72871]: ts=2024-09-11T08:08:15.882Z caller=node_exporter.go:117 level=info collector=nvme 2024-09-11T08:08:16.182 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Sep 11 08:08:15 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-a[72871]: ts=2024-09-11T08:08:15.882Z caller=node_exporter.go:117 level=info collector=os 2024-09-11T08:08:16.182 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Sep 11 08:08:15 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-a[72871]: ts=2024-09-11T08:08:15.882Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-09-11T08:08:16.182 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Sep 11 08:08:15 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-a[72871]: ts=2024-09-11T08:08:15.882Z caller=node_exporter.go:117 level=info collector=pressure 2024-09-11T08:08:16.182 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Sep 11 08:08:15 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-a[72871]: ts=2024-09-11T08:08:15.882Z caller=node_exporter.go:117 level=info collector=rapl 2024-09-11T08:08:16.182 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Sep 11 08:08:15 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-a[72871]: ts=2024-09-11T08:08:15.882Z caller=node_exporter.go:117 level=info collector=schedstat 2024-09-11T08:08:16.182 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Sep 11 08:08:15 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-a[72871]: ts=2024-09-11T08:08:15.882Z caller=node_exporter.go:117 level=info collector=selinux 2024-09-11T08:08:16.182 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Sep 11 08:08:15 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-a[72871]: ts=2024-09-11T08:08:15.882Z caller=node_exporter.go:117 level=info collector=sockstat 2024-09-11T08:08:16.182 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Sep 11 08:08:15 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-a[72871]: ts=2024-09-11T08:08:15.882Z caller=node_exporter.go:117 level=info collector=softnet 2024-09-11T08:08:16.183 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Sep 11 08:08:15 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-a[72871]: ts=2024-09-11T08:08:15.882Z caller=node_exporter.go:117 level=info collector=stat 2024-09-11T08:08:16.183 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Sep 11 08:08:15 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-a[72871]: ts=2024-09-11T08:08:15.882Z caller=node_exporter.go:117 level=info collector=tapestats 2024-09-11T08:08:16.183 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Sep 11 08:08:15 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-a[72871]: ts=2024-09-11T08:08:15.882Z caller=node_exporter.go:117 level=info collector=textfile 2024-09-11T08:08:16.183 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Sep 11 08:08:15 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-a[72871]: ts=2024-09-11T08:08:15.882Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-09-11T08:08:16.183 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Sep 11 08:08:15 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-a[72871]: ts=2024-09-11T08:08:15.882Z caller=node_exporter.go:117 level=info collector=time 2024-09-11T08:08:16.183 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Sep 11 08:08:15 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-a[72871]: ts=2024-09-11T08:08:15.882Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-09-11T08:08:16.183 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Sep 11 08:08:15 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-a[72871]: ts=2024-09-11T08:08:15.882Z caller=node_exporter.go:117 level=info collector=uname 2024-09-11T08:08:16.183 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Sep 11 08:08:15 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-a[72871]: ts=2024-09-11T08:08:15.882Z caller=node_exporter.go:117 level=info collector=vmstat 2024-09-11T08:08:16.183 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Sep 11 08:08:15 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-a[72871]: ts=2024-09-11T08:08:15.882Z caller=node_exporter.go:117 level=info collector=xfs 2024-09-11T08:08:16.183 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Sep 11 08:08:15 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-a[72871]: ts=2024-09-11T08:08:15.882Z caller=node_exporter.go:117 level=info collector=zfs 2024-09-11T08:08:16.183 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Sep 11 08:08:15 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-a[72871]: ts=2024-09-11T08:08:15.883Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-09-11T08:08:16.183 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Sep 11 08:08:15 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-a[72871]: ts=2024-09-11T08:08:15.883Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-09-11T08:08:16.183 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Sep 11 08:08:15 smithi122 bash[72771]: f25b60c5acc6bffcf0683390d9d9726c724df1582a7c4d3fc840425271e6db01 2024-09-11T08:08:16.183 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Sep 11 08:08:15 smithi122 systemd[1]: Started Ceph node-exporter.a for f1d75850-7013-11ef-bcea-c7b262605968. 2024-09-11T08:08:16.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:16 smithi135 ceph-mon[32689]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:08:16.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:16 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:16.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:16 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:16.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:16 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:16.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:16 smithi122 ceph-mon[28992]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:08:16.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:16 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:16.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:16 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:16.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:16 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:16.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:16 smithi122 ceph-mon[39726]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:08:16.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:16 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:16.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:16 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:16.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:16 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:16.946 INFO:journalctl@ceph.node-exporter.b.smithi135.stdout:Sep 11 08:08:16 smithi135 systemd[1]: Starting Ceph node-exporter.b for f1d75850-7013-11ef-bcea-c7b262605968... 2024-09-11T08:08:17.228 INFO:journalctl@ceph.node-exporter.b.smithi135.stdout:Sep 11 08:08:16 smithi135 bash[63520]: Trying to pull quay.io/prometheus/node-exporter:v1.5.0... 2024-09-11T08:08:17.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:17 smithi135 ceph-mon[32689]: Deploying daemon node-exporter.b on smithi135 2024-09-11T08:08:17.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:17 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 2]}]: dispatch 2024-09-11T08:08:17.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:17 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2024-09-11T08:08:17.229 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:17 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 2]}]: dispatch 2024-09-11T08:08:17.229 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:17 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2024-09-11T08:08:17.229 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:17 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2024-09-11T08:08:17.229 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:17 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2024-09-11T08:08:17.229 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:17 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2024-09-11T08:08:17.229 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:17 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2024-09-11T08:08:17.229 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:17 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:08:17.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:17 smithi122 ceph-mon[28992]: Deploying daemon node-exporter.b on smithi135 2024-09-11T08:08:17.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:17 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 2]}]: dispatch 2024-09-11T08:08:17.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:17 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2024-09-11T08:08:17.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:17 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 2]}]: dispatch 2024-09-11T08:08:17.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:17 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2024-09-11T08:08:17.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:17 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2024-09-11T08:08:17.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:17 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2024-09-11T08:08:17.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:17 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2024-09-11T08:08:17.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:17 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2024-09-11T08:08:17.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:17 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:08:17.486 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:17 smithi122 ceph-mon[39726]: Deploying daemon node-exporter.b on smithi135 2024-09-11T08:08:17.486 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:17 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 2]}]: dispatch 2024-09-11T08:08:17.486 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:17 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2024-09-11T08:08:17.486 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:17 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 2]}]: dispatch 2024-09-11T08:08:17.486 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:17 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2024-09-11T08:08:17.486 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:17 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2024-09-11T08:08:17.486 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:17 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2024-09-11T08:08:17.487 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:17 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2024-09-11T08:08:17.487 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:17 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2024-09-11T08:08:17.487 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:17 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:08:17.728 INFO:journalctl@ceph.node-exporter.b.smithi135.stdout:Sep 11 08:08:17 smithi135 bash[63520]: Getting image source signatures 2024-09-11T08:08:17.728 INFO:journalctl@ceph.node-exporter.b.smithi135.stdout:Sep 11 08:08:17 smithi135 bash[63520]: Copying blob sha256:c0e87333d3805a7b7e1c25783dd25f27da0fe70a8e262eee7ed190c8dc843fa7 2024-09-11T08:08:17.728 INFO:journalctl@ceph.node-exporter.b.smithi135.stdout:Sep 11 08:08:17 smithi135 bash[63520]: Copying blob sha256:22b70bddd3acadc892fca4c2af4260629bfda5dfd11ebc106a93ce24e752b5ed 2024-09-11T08:08:17.729 INFO:journalctl@ceph.node-exporter.b.smithi135.stdout:Sep 11 08:08:17 smithi135 bash[63520]: Copying blob sha256:5c12815fee558b157dee7f7509dedbaba0a8379098858a65ec869e1f1526ea0c 2024-09-11T08:08:18.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:18 smithi135 ceph-mon[32689]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:08:18.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:18 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 2]}]': finished 2024-09-11T08:08:18.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:18 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]': finished 2024-09-11T08:08:18.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:18 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]': finished 2024-09-11T08:08:18.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:18 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]': finished 2024-09-11T08:08:18.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:18 smithi135 ceph-mon[32689]: osdmap e62: 8 total, 8 up, 8 in 2024-09-11T08:08:18.229 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:08:17 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: ::ffff:172.21.15.135 - - [11/Sep/2024:08:08:17] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:08:18.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:18 smithi122 ceph-mon[39726]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:08:18.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:18 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 2]}]': finished 2024-09-11T08:08:18.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:18 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]': finished 2024-09-11T08:08:18.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:18 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]': finished 2024-09-11T08:08:18.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:18 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]': finished 2024-09-11T08:08:18.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:18 smithi122 ceph-mon[39726]: osdmap e62: 8 total, 8 up, 8 in 2024-09-11T08:08:18.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:18 smithi122 ceph-mon[28992]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:08:18.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:18 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 2]}]': finished 2024-09-11T08:08:18.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:18 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]': finished 2024-09-11T08:08:18.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:18 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]': finished 2024-09-11T08:08:18.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:18 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]': finished 2024-09-11T08:08:18.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:18 smithi122 ceph-mon[28992]: osdmap e62: 8 total, 8 up, 8 in 2024-09-11T08:08:18.801 INFO:teuthology.orchestra.run.smithi122.stderr:++ ceph auth get-key osd.0 2024-09-11T08:08:18.978 INFO:journalctl@ceph.node-exporter.b.smithi135.stdout:Sep 11 08:08:18 smithi135 bash[63520]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-09-11T08:08:18.978 INFO:journalctl@ceph.node-exporter.b.smithi135.stdout:Sep 11 08:08:18 smithi135 bash[63520]: Writing manifest to image destination 2024-09-11T08:08:19.118 INFO:teuthology.orchestra.run.smithi122.stderr:+ NK=AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== 2024-09-11T08:08:19.118 INFO:teuthology.orchestra.run.smithi122.stderr:+ '[' AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== == AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== ']' 2024-09-11T08:08:19.118 INFO:teuthology.orchestra.run.smithi122.stderr:+ sleep 5 2024-09-11T08:08:19.324 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:19 smithi135 ceph-mon[32689]: osdmap e63: 8 total, 8 up, 8 in 2024-09-11T08:08:19.325 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:19 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/3479684552' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:08:19.325 INFO:journalctl@ceph.node-exporter.b.smithi135.stdout:Sep 11 08:08:19 smithi135 podman[63520]: 2024-09-11 08:08:19.045912548 +0000 UTC m=+2.194658870 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-09-11T08:08:19.325 INFO:journalctl@ceph.node-exporter.b.smithi135.stdout:Sep 11 08:08:19 smithi135 podman[63520]: 2024-09-11 08:08:19.14451404 +0000 UTC m=+2.293260401 container create 17ab6fbddd24434760d64d36180e166e0509bc08557b2055cb345b6947848323 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-11T08:08:19.325 INFO:journalctl@ceph.node-exporter.b.smithi135.stdout:Sep 11 08:08:19 smithi135 podman[63520]: 2024-09-11 08:08:19.321520386 +0000 UTC m=+2.470266705 container init 17ab6fbddd24434760d64d36180e166e0509bc08557b2055cb345b6947848323 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-11T08:08:19.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:19 smithi122 ceph-mon[39726]: osdmap e63: 8 total, 8 up, 8 in 2024-09-11T08:08:19.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:19 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/3479684552' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:08:19.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:19 smithi122 ceph-mon[28992]: osdmap e63: 8 total, 8 up, 8 in 2024-09-11T08:08:19.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:19 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/3479684552' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:08:19.728 INFO:journalctl@ceph.node-exporter.b.smithi135.stdout:Sep 11 08:08:19 smithi135 podman[63520]: 2024-09-11 08:08:19.324551881 +0000 UTC m=+2.473298203 container start 17ab6fbddd24434760d64d36180e166e0509bc08557b2055cb345b6947848323 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-11T08:08:19.728 INFO:journalctl@ceph.node-exporter.b.smithi135.stdout:Sep 11 08:08:19 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-b[63595]: ts=2024-09-11T08:08:19.328Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-09-11T08:08:19.728 INFO:journalctl@ceph.node-exporter.b.smithi135.stdout:Sep 11 08:08:19 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-b[63595]: ts=2024-09-11T08:08:19.328Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-09-11T08:08:19.729 INFO:journalctl@ceph.node-exporter.b.smithi135.stdout:Sep 11 08:08:19 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-b[63595]: ts=2024-09-11T08:08:19.329Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-09-11T08:08:19.729 INFO:journalctl@ceph.node-exporter.b.smithi135.stdout:Sep 11 08:08:19 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-b[63595]: ts=2024-09-11T08:08:19.329Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-09-11T08:08:19.729 INFO:journalctl@ceph.node-exporter.b.smithi135.stdout:Sep 11 08:08:19 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-b[63595]: ts=2024-09-11T08:08:19.329Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-09-11T08:08:19.729 INFO:journalctl@ceph.node-exporter.b.smithi135.stdout:Sep 11 08:08:19 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-b[63595]: ts=2024-09-11T08:08:19.329Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-09-11T08:08:19.729 INFO:journalctl@ceph.node-exporter.b.smithi135.stdout:Sep 11 08:08:19 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-b[63595]: ts=2024-09-11T08:08:19.329Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-09-11T08:08:19.729 INFO:journalctl@ceph.node-exporter.b.smithi135.stdout:Sep 11 08:08:19 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-b[63595]: ts=2024-09-11T08:08:19.329Z caller=node_exporter.go:117 level=info collector=arp 2024-09-11T08:08:19.729 INFO:journalctl@ceph.node-exporter.b.smithi135.stdout:Sep 11 08:08:19 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-b[63595]: ts=2024-09-11T08:08:19.329Z caller=node_exporter.go:117 level=info collector=bcache 2024-09-11T08:08:19.729 INFO:journalctl@ceph.node-exporter.b.smithi135.stdout:Sep 11 08:08:19 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-b[63595]: ts=2024-09-11T08:08:19.329Z caller=node_exporter.go:117 level=info collector=bonding 2024-09-11T08:08:19.729 INFO:journalctl@ceph.node-exporter.b.smithi135.stdout:Sep 11 08:08:19 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-b[63595]: ts=2024-09-11T08:08:19.329Z caller=node_exporter.go:117 level=info collector=btrfs 2024-09-11T08:08:19.729 INFO:journalctl@ceph.node-exporter.b.smithi135.stdout:Sep 11 08:08:19 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-b[63595]: ts=2024-09-11T08:08:19.329Z caller=node_exporter.go:117 level=info collector=conntrack 2024-09-11T08:08:19.729 INFO:journalctl@ceph.node-exporter.b.smithi135.stdout:Sep 11 08:08:19 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-b[63595]: ts=2024-09-11T08:08:19.329Z caller=node_exporter.go:117 level=info collector=cpu 2024-09-11T08:08:19.729 INFO:journalctl@ceph.node-exporter.b.smithi135.stdout:Sep 11 08:08:19 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-b[63595]: ts=2024-09-11T08:08:19.329Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-09-11T08:08:19.729 INFO:journalctl@ceph.node-exporter.b.smithi135.stdout:Sep 11 08:08:19 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-b[63595]: ts=2024-09-11T08:08:19.329Z caller=node_exporter.go:117 level=info collector=diskstats 2024-09-11T08:08:19.730 INFO:journalctl@ceph.node-exporter.b.smithi135.stdout:Sep 11 08:08:19 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-b[63595]: ts=2024-09-11T08:08:19.329Z caller=node_exporter.go:117 level=info collector=dmi 2024-09-11T08:08:19.730 INFO:journalctl@ceph.node-exporter.b.smithi135.stdout:Sep 11 08:08:19 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-b[63595]: ts=2024-09-11T08:08:19.329Z caller=node_exporter.go:117 level=info collector=edac 2024-09-11T08:08:19.730 INFO:journalctl@ceph.node-exporter.b.smithi135.stdout:Sep 11 08:08:19 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-b[63595]: ts=2024-09-11T08:08:19.329Z caller=node_exporter.go:117 level=info collector=entropy 2024-09-11T08:08:19.730 INFO:journalctl@ceph.node-exporter.b.smithi135.stdout:Sep 11 08:08:19 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-b[63595]: ts=2024-09-11T08:08:19.329Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-09-11T08:08:19.730 INFO:journalctl@ceph.node-exporter.b.smithi135.stdout:Sep 11 08:08:19 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-b[63595]: ts=2024-09-11T08:08:19.329Z caller=node_exporter.go:117 level=info collector=filefd 2024-09-11T08:08:19.730 INFO:journalctl@ceph.node-exporter.b.smithi135.stdout:Sep 11 08:08:19 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-b[63595]: ts=2024-09-11T08:08:19.329Z caller=node_exporter.go:117 level=info collector=filesystem 2024-09-11T08:08:19.730 INFO:journalctl@ceph.node-exporter.b.smithi135.stdout:Sep 11 08:08:19 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-b[63595]: ts=2024-09-11T08:08:19.330Z caller=node_exporter.go:117 level=info collector=hwmon 2024-09-11T08:08:19.731 INFO:journalctl@ceph.node-exporter.b.smithi135.stdout:Sep 11 08:08:19 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-b[63595]: ts=2024-09-11T08:08:19.330Z caller=node_exporter.go:117 level=info collector=infiniband 2024-09-11T08:08:19.731 INFO:journalctl@ceph.node-exporter.b.smithi135.stdout:Sep 11 08:08:19 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-b[63595]: ts=2024-09-11T08:08:19.330Z caller=node_exporter.go:117 level=info collector=ipvs 2024-09-11T08:08:19.731 INFO:journalctl@ceph.node-exporter.b.smithi135.stdout:Sep 11 08:08:19 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-b[63595]: ts=2024-09-11T08:08:19.330Z caller=node_exporter.go:117 level=info collector=loadavg 2024-09-11T08:08:19.731 INFO:journalctl@ceph.node-exporter.b.smithi135.stdout:Sep 11 08:08:19 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-b[63595]: ts=2024-09-11T08:08:19.330Z caller=node_exporter.go:117 level=info collector=mdadm 2024-09-11T08:08:19.731 INFO:journalctl@ceph.node-exporter.b.smithi135.stdout:Sep 11 08:08:19 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-b[63595]: ts=2024-09-11T08:08:19.330Z caller=node_exporter.go:117 level=info collector=meminfo 2024-09-11T08:08:19.731 INFO:journalctl@ceph.node-exporter.b.smithi135.stdout:Sep 11 08:08:19 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-b[63595]: ts=2024-09-11T08:08:19.330Z caller=node_exporter.go:117 level=info collector=netclass 2024-09-11T08:08:19.731 INFO:journalctl@ceph.node-exporter.b.smithi135.stdout:Sep 11 08:08:19 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-b[63595]: ts=2024-09-11T08:08:19.330Z caller=node_exporter.go:117 level=info collector=netdev 2024-09-11T08:08:19.731 INFO:journalctl@ceph.node-exporter.b.smithi135.stdout:Sep 11 08:08:19 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-b[63595]: ts=2024-09-11T08:08:19.330Z caller=node_exporter.go:117 level=info collector=netstat 2024-09-11T08:08:19.731 INFO:journalctl@ceph.node-exporter.b.smithi135.stdout:Sep 11 08:08:19 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-b[63595]: ts=2024-09-11T08:08:19.330Z caller=node_exporter.go:117 level=info collector=nfs 2024-09-11T08:08:19.731 INFO:journalctl@ceph.node-exporter.b.smithi135.stdout:Sep 11 08:08:19 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-b[63595]: ts=2024-09-11T08:08:19.330Z caller=node_exporter.go:117 level=info collector=nfsd 2024-09-11T08:08:19.731 INFO:journalctl@ceph.node-exporter.b.smithi135.stdout:Sep 11 08:08:19 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-b[63595]: ts=2024-09-11T08:08:19.330Z caller=node_exporter.go:117 level=info collector=nvme 2024-09-11T08:08:19.731 INFO:journalctl@ceph.node-exporter.b.smithi135.stdout:Sep 11 08:08:19 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-b[63595]: ts=2024-09-11T08:08:19.330Z caller=node_exporter.go:117 level=info collector=os 2024-09-11T08:08:19.731 INFO:journalctl@ceph.node-exporter.b.smithi135.stdout:Sep 11 08:08:19 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-b[63595]: ts=2024-09-11T08:08:19.330Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-09-11T08:08:19.732 INFO:journalctl@ceph.node-exporter.b.smithi135.stdout:Sep 11 08:08:19 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-b[63595]: ts=2024-09-11T08:08:19.330Z caller=node_exporter.go:117 level=info collector=pressure 2024-09-11T08:08:19.732 INFO:journalctl@ceph.node-exporter.b.smithi135.stdout:Sep 11 08:08:19 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-b[63595]: ts=2024-09-11T08:08:19.330Z caller=node_exporter.go:117 level=info collector=rapl 2024-09-11T08:08:19.732 INFO:journalctl@ceph.node-exporter.b.smithi135.stdout:Sep 11 08:08:19 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-b[63595]: ts=2024-09-11T08:08:19.330Z caller=node_exporter.go:117 level=info collector=schedstat 2024-09-11T08:08:19.732 INFO:journalctl@ceph.node-exporter.b.smithi135.stdout:Sep 11 08:08:19 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-b[63595]: ts=2024-09-11T08:08:19.330Z caller=node_exporter.go:117 level=info collector=selinux 2024-09-11T08:08:19.732 INFO:journalctl@ceph.node-exporter.b.smithi135.stdout:Sep 11 08:08:19 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-b[63595]: ts=2024-09-11T08:08:19.330Z caller=node_exporter.go:117 level=info collector=sockstat 2024-09-11T08:08:19.732 INFO:journalctl@ceph.node-exporter.b.smithi135.stdout:Sep 11 08:08:19 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-b[63595]: ts=2024-09-11T08:08:19.330Z caller=node_exporter.go:117 level=info collector=softnet 2024-09-11T08:08:19.732 INFO:journalctl@ceph.node-exporter.b.smithi135.stdout:Sep 11 08:08:19 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-b[63595]: ts=2024-09-11T08:08:19.330Z caller=node_exporter.go:117 level=info collector=stat 2024-09-11T08:08:19.732 INFO:journalctl@ceph.node-exporter.b.smithi135.stdout:Sep 11 08:08:19 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-b[63595]: ts=2024-09-11T08:08:19.330Z caller=node_exporter.go:117 level=info collector=tapestats 2024-09-11T08:08:19.732 INFO:journalctl@ceph.node-exporter.b.smithi135.stdout:Sep 11 08:08:19 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-b[63595]: ts=2024-09-11T08:08:19.330Z caller=node_exporter.go:117 level=info collector=textfile 2024-09-11T08:08:19.732 INFO:journalctl@ceph.node-exporter.b.smithi135.stdout:Sep 11 08:08:19 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-b[63595]: ts=2024-09-11T08:08:19.330Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-09-11T08:08:19.732 INFO:journalctl@ceph.node-exporter.b.smithi135.stdout:Sep 11 08:08:19 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-b[63595]: ts=2024-09-11T08:08:19.330Z caller=node_exporter.go:117 level=info collector=time 2024-09-11T08:08:19.732 INFO:journalctl@ceph.node-exporter.b.smithi135.stdout:Sep 11 08:08:19 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-b[63595]: ts=2024-09-11T08:08:19.330Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-09-11T08:08:19.732 INFO:journalctl@ceph.node-exporter.b.smithi135.stdout:Sep 11 08:08:19 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-b[63595]: ts=2024-09-11T08:08:19.330Z caller=node_exporter.go:117 level=info collector=uname 2024-09-11T08:08:19.732 INFO:journalctl@ceph.node-exporter.b.smithi135.stdout:Sep 11 08:08:19 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-b[63595]: ts=2024-09-11T08:08:19.330Z caller=node_exporter.go:117 level=info collector=vmstat 2024-09-11T08:08:19.733 INFO:journalctl@ceph.node-exporter.b.smithi135.stdout:Sep 11 08:08:19 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-b[63595]: ts=2024-09-11T08:08:19.330Z caller=node_exporter.go:117 level=info collector=xfs 2024-09-11T08:08:19.733 INFO:journalctl@ceph.node-exporter.b.smithi135.stdout:Sep 11 08:08:19 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-b[63595]: ts=2024-09-11T08:08:19.330Z caller=node_exporter.go:117 level=info collector=zfs 2024-09-11T08:08:19.733 INFO:journalctl@ceph.node-exporter.b.smithi135.stdout:Sep 11 08:08:19 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-b[63595]: ts=2024-09-11T08:08:19.330Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-09-11T08:08:19.733 INFO:journalctl@ceph.node-exporter.b.smithi135.stdout:Sep 11 08:08:19 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-b[63595]: ts=2024-09-11T08:08:19.330Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-09-11T08:08:19.733 INFO:journalctl@ceph.node-exporter.b.smithi135.stdout:Sep 11 08:08:19 smithi135 bash[63520]: 17ab6fbddd24434760d64d36180e166e0509bc08557b2055cb345b6947848323 2024-09-11T08:08:19.733 INFO:journalctl@ceph.node-exporter.b.smithi135.stdout:Sep 11 08:08:19 smithi135 systemd[1]: Started Ceph node-exporter.b for f1d75850-7013-11ef-bcea-c7b262605968. 2024-09-11T08:08:20.409 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:20 smithi135 ceph-mon[32689]: pgmap v36: 132 pgs: 4 peering, 128 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:08:20.409 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:20 smithi135 ceph-mon[32689]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-11T08:08:20.409 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:20 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:20.409 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:20 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:20.409 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:20 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:20.409 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:20 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:20.410 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:20 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:20.410 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:20 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:08:20.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:20 smithi122 ceph-mon[39726]: pgmap v36: 132 pgs: 4 peering, 128 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:08:20.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:20 smithi122 ceph-mon[39726]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-11T08:08:20.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:20 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:20.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:20 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:20.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:20 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:20.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:20 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:20.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:20 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:20.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:20 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:08:20.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:20 smithi122 ceph-mon[28992]: pgmap v36: 132 pgs: 4 peering, 128 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:08:20.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:20 smithi122 ceph-mon[28992]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-11T08:08:20.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:20 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:20.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:20 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:20.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:20 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:20.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:20 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:20.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:20 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:20.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:20 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:08:22.199 INFO:journalctl@ceph.iscsi.iscsi.a.smithi135.stdout:Sep 11 08:08:21 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[59799]: debug there is no tcmu-runner data available 2024-09-11T08:08:22.337 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:22 smithi122 ceph-mon[39726]: pgmap v37: 132 pgs: 4 peering, 128 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:08:22.337 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:22 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:22.337 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:22 smithi122 ceph-mon[28992]: pgmap v37: 132 pgs: 4 peering, 128 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:08:22.337 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:22 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:22.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:22 smithi135 ceph-mon[32689]: pgmap v37: 132 pgs: 4 peering, 128 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:08:22.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:22 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:23.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:23 smithi135 ceph-mon[32689]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:08:23.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:23 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:23.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:23 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:23.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:23 smithi122 ceph-mon[28992]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:08:23.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:23 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:23.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:23 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:23.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:23 smithi122 ceph-mon[39726]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:08:23.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:23 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:23.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:23 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:24.120 INFO:teuthology.orchestra.run.smithi122.stderr:++ ceph auth get-key osd.0 2024-09-11T08:08:24.449 INFO:teuthology.orchestra.run.smithi122.stderr:+ NK=AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== 2024-09-11T08:08:24.450 INFO:teuthology.orchestra.run.smithi122.stderr:+ '[' AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== == AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== ']' 2024-09-11T08:08:24.450 INFO:teuthology.orchestra.run.smithi122.stderr:+ sleep 5 2024-09-11T08:08:24.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:24 smithi122 ceph-mon[28992]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 72 B/s, 2 objects/s recovering 2024-09-11T08:08:24.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:24 smithi122 ceph-mon[28992]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-11T08:08:24.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:24 smithi122 ceph-mon[28992]: Cluster is now healthy 2024-09-11T08:08:24.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:24 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:24.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:24 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:24.735 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:24 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:08:24.735 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:24 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:08:24.735 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:24 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:24.735 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:24 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.0", "format": "json"}]: dispatch 2024-09-11T08:08:24.735 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:24 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.0", "format": "json"}]: dispatch 2024-09-11T08:08:24.735 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:24 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.0", "format": "json"}]': finished 2024-09-11T08:08:24.735 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:24 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/2532248184' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:08:24.735 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:24 smithi122 ceph-mon[39726]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 72 B/s, 2 objects/s recovering 2024-09-11T08:08:24.735 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:24 smithi122 ceph-mon[39726]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-11T08:08:24.736 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:24 smithi122 ceph-mon[39726]: Cluster is now healthy 2024-09-11T08:08:24.736 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:24 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:24.736 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:24 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:24.736 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:24 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:08:24.736 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:24 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:08:24.736 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:24 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:24.736 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:24 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.0", "format": "json"}]: dispatch 2024-09-11T08:08:24.736 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:24 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.0", "format": "json"}]: dispatch 2024-09-11T08:08:24.736 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:24 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.0", "format": "json"}]': finished 2024-09-11T08:08:24.736 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:24 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/2532248184' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:08:24.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:24 smithi135 ceph-mon[32689]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 72 B/s, 2 objects/s recovering 2024-09-11T08:08:24.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:24 smithi135 ceph-mon[32689]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-11T08:08:24.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:24 smithi135 ceph-mon[32689]: Cluster is now healthy 2024-09-11T08:08:24.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:24 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:24.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:24 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:24.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:24 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:08:24.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:24 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:08:24.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:24 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:24.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:24 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.0", "format": "json"}]: dispatch 2024-09-11T08:08:24.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:24 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.0", "format": "json"}]: dispatch 2024-09-11T08:08:24.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:24 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.0", "format": "json"}]': finished 2024-09-11T08:08:24.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:24 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/2532248184' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:08:25.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:25 smithi135 ceph-mon[32689]: Rotating authentication key for osd.0 2024-09-11T08:08:25.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:25 smithi135 ceph-mon[32689]: Reconfiguring daemon osd.0 on smithi122 2024-09-11T08:08:25.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:25 smithi122 ceph-mon[39726]: Rotating authentication key for osd.0 2024-09-11T08:08:25.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:25 smithi122 ceph-mon[39726]: Reconfiguring daemon osd.0 on smithi122 2024-09-11T08:08:25.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:25 smithi122 ceph-mon[28992]: Rotating authentication key for osd.0 2024-09-11T08:08:25.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:25 smithi122 ceph-mon[28992]: Reconfiguring daemon osd.0 on smithi122 2024-09-11T08:08:26.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:26 smithi122 ceph-mon[39726]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 72 B/s, 2 objects/s recovering 2024-09-11T08:08:26.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:26 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:26.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:26 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:26.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:26 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:26.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:26 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:26.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:26 smithi122 ceph-mon[28992]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 72 B/s, 2 objects/s recovering 2024-09-11T08:08:26.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:26 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:26.735 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:26 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:26.735 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:26 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:26.735 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:26 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:26.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:26 smithi135 ceph-mon[32689]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 72 B/s, 2 objects/s recovering 2024-09-11T08:08:26.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:26 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:26.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:26 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:26.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:26 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:26.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:26 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:27.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:27 smithi122 ceph-mon[28992]: Reconfiguring alertmanager.a (dependencies changed)... 2024-09-11T08:08:27.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:27 smithi122 ceph-mon[28992]: Reconfiguring daemon alertmanager.a on smithi122 2024-09-11T08:08:27.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:27 smithi122 ceph-mon[39726]: Reconfiguring alertmanager.a (dependencies changed)... 2024-09-11T08:08:27.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:27 smithi122 ceph-mon[39726]: Reconfiguring daemon alertmanager.a on smithi122 2024-09-11T08:08:27.935 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:27 smithi135 ceph-mon[32689]: Reconfiguring alertmanager.a (dependencies changed)... 2024-09-11T08:08:27.935 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:27 smithi135 ceph-mon[32689]: Reconfiguring daemon alertmanager.a on smithi122 2024-09-11T08:08:28.228 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:08:27 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: ::ffff:172.21.15.135 - - [11/Sep/2024:08:08:27] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:08:28.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:28 smithi122 ceph-mon[28992]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 931 B/s rd, 0 op/s; 58 B/s, 2 objects/s recovering 2024-09-11T08:08:28.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:28 smithi122 ceph-mon[39726]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 931 B/s rd, 0 op/s; 58 B/s, 2 objects/s recovering 2024-09-11T08:08:28.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:28 smithi135 ceph-mon[32689]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 931 B/s rd, 0 op/s; 58 B/s, 2 objects/s recovering 2024-09-11T08:08:29.390 INFO:journalctl@ceph.alertmanager.a.smithi122.stdout:Sep 11 08:08:29 smithi122 systemd[1]: Stopping Ceph alertmanager.a for f1d75850-7013-11ef-bcea-c7b262605968... 2024-09-11T08:08:29.453 INFO:teuthology.orchestra.run.smithi122.stderr:++ ceph auth get-key osd.0 2024-09-11T08:08:29.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:29 smithi122 ceph-mon[28992]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 5.5 KiB/s rd, 470 B/s wr, 11 op/s; 53 B/s, 2 objects/s recovering 2024-09-11T08:08:29.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:29 smithi122 ceph-mon[39726]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 5.5 KiB/s rd, 470 B/s wr, 11 op/s; 53 B/s, 2 objects/s recovering 2024-09-11T08:08:29.735 INFO:journalctl@ceph.alertmanager.a.smithi122.stdout:Sep 11 08:08:29 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-alertmanager-a[71755]: ts=2024-09-11T08:08:29.523Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-09-11T08:08:29.735 INFO:journalctl@ceph.alertmanager.a.smithi122.stdout:Sep 11 08:08:29 smithi122 podman[73816]: 2024-09-11 08:08:29.543994354 +0000 UTC m=+0.142203642 container died fd8a7d7645ec55289f3793e0212735aee458d54e607812ec1786f1d7e6c2a575 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T08:08:29.776 INFO:teuthology.orchestra.run.smithi122.stderr:+ NK=AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== 2024-09-11T08:08:29.776 INFO:teuthology.orchestra.run.smithi122.stderr:+ '[' AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== == AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== ']' 2024-09-11T08:08:29.777 INFO:teuthology.orchestra.run.smithi122.stderr:+ sleep 5 2024-09-11T08:08:29.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:29 smithi135 ceph-mon[32689]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 5.5 KiB/s rd, 470 B/s wr, 11 op/s; 53 B/s, 2 objects/s recovering 2024-09-11T08:08:30.442 INFO:journalctl@ceph.alertmanager.a.smithi122.stdout:Sep 11 08:08:30 smithi122 podman[73816]: 2024-09-11 08:08:30.040176686 +0000 UTC m=+0.638385973 container cleanup fd8a7d7645ec55289f3793e0212735aee458d54e607812ec1786f1d7e6c2a575 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T08:08:30.442 INFO:journalctl@ceph.alertmanager.a.smithi122.stdout:Sep 11 08:08:30 smithi122 bash[73816]: ceph-f1d75850-7013-11ef-bcea-c7b262605968-alertmanager-a 2024-09-11T08:08:30.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:30 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/3250645702' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:08:30.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:30 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/3250645702' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:08:30.734 INFO:journalctl@ceph.alertmanager.a.smithi122.stdout:Sep 11 08:08:30 smithi122 podman[73848]: 2024-09-11 08:08:30.442076979 +0000 UTC m=+0.892994958 container remove fd8a7d7645ec55289f3793e0212735aee458d54e607812ec1786f1d7e6c2a575 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T08:08:30.735 INFO:journalctl@ceph.alertmanager.a.smithi122.stdout:Sep 11 08:08:30 smithi122 podman[73848]: 2024-09-11 08:08:30.525838476 +0000 UTC m=+0.976756460 volume remove 721486e6d180ff57b5a604d355a1a72dd90e7d824a53d8095a78be70766bea38 2024-09-11T08:08:30.735 INFO:journalctl@ceph.alertmanager.a.smithi122.stdout:Sep 11 08:08:30 smithi122 systemd[1]: ceph-f1d75850-7013-11ef-bcea-c7b262605968@alertmanager.a.service: Deactivated successfully. 2024-09-11T08:08:30.735 INFO:journalctl@ceph.alertmanager.a.smithi122.stdout:Sep 11 08:08:30 smithi122 systemd[1]: Stopped Ceph alertmanager.a for f1d75850-7013-11ef-bcea-c7b262605968. 2024-09-11T08:08:30.735 INFO:journalctl@ceph.alertmanager.a.smithi122.stdout:Sep 11 08:08:30 smithi122 systemd[1]: Starting Ceph alertmanager.a for f1d75850-7013-11ef-bcea-c7b262605968... 2024-09-11T08:08:30.879 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:30 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/3250645702' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:08:31.167 INFO:journalctl@ceph.alertmanager.a.smithi122.stdout:Sep 11 08:08:30 smithi122 podman[73943]: 2024-09-11 08:08:30.762090994 +0000 UTC m=+0.016064530 image pull c8568f914cd25b2062c44e9f79f9c18da6e3b85fe0c47a12a2191c61426c2b19 quay.io/prometheus/alertmanager:v0.25.0 2024-09-11T08:08:31.167 INFO:journalctl@ceph.alertmanager.a.smithi122.stdout:Sep 11 08:08:30 smithi122 podman[73943]: 2024-09-11 08:08:30.860096493 +0000 UTC m=+0.114070030 volume create 4e806b6924b6173ff4daa3c7ad4ac858a68c739540b5e97ff8b59a165fb78a88 2024-09-11T08:08:31.168 INFO:journalctl@ceph.alertmanager.a.smithi122.stdout:Sep 11 08:08:30 smithi122 podman[73943]: 2024-09-11 08:08:30.977223923 +0000 UTC m=+0.231197459 container create bb573119c40ea6a5709763a232d56434b0fc14318b965be0a5f85aae27fb04ae (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T08:08:31.734 INFO:journalctl@ceph.alertmanager.a.smithi122.stdout:Sep 11 08:08:31 smithi122 podman[73943]: 2024-09-11 08:08:31.271911242 +0000 UTC m=+0.525884777 container init bb573119c40ea6a5709763a232d56434b0fc14318b965be0a5f85aae27fb04ae (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T08:08:31.734 INFO:journalctl@ceph.alertmanager.a.smithi122.stdout:Sep 11 08:08:31 smithi122 podman[73943]: 2024-09-11 08:08:31.274675494 +0000 UTC m=+0.528649039 container start bb573119c40ea6a5709763a232d56434b0fc14318b965be0a5f85aae27fb04ae (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T08:08:31.734 INFO:journalctl@ceph.alertmanager.a.smithi122.stdout:Sep 11 08:08:31 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-alertmanager-a[73957]: ts=2024-09-11T08:08:31.287Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=HEAD, revision=258fab7cdd551f2cf251ed0348f0ad7289aee789)" 2024-09-11T08:08:31.735 INFO:journalctl@ceph.alertmanager.a.smithi122.stdout:Sep 11 08:08:31 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-alertmanager-a[73957]: ts=2024-09-11T08:08:31.287Z caller=main.go:241 level=info build_context="(go=go1.19.4, user=root@abe866dd5717, date=20221222-14:51:36)" 2024-09-11T08:08:31.735 INFO:journalctl@ceph.alertmanager.a.smithi122.stdout:Sep 11 08:08:31 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-alertmanager-a[73957]: ts=2024-09-11T08:08:31.288Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.122 port=9094 2024-09-11T08:08:31.735 INFO:journalctl@ceph.alertmanager.a.smithi122.stdout:Sep 11 08:08:31 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-alertmanager-a[73957]: ts=2024-09-11T08:08:31.289Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-09-11T08:08:31.735 INFO:journalctl@ceph.alertmanager.a.smithi122.stdout:Sep 11 08:08:31 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-alertmanager-a[73957]: ts=2024-09-11T08:08:31.311Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-11T08:08:31.735 INFO:journalctl@ceph.alertmanager.a.smithi122.stdout:Sep 11 08:08:31 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-alertmanager-a[73957]: ts=2024-09-11T08:08:31.311Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-11T08:08:31.735 INFO:journalctl@ceph.alertmanager.a.smithi122.stdout:Sep 11 08:08:31 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-alertmanager-a[73957]: ts=2024-09-11T08:08:31.315Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9093 2024-09-11T08:08:31.735 INFO:journalctl@ceph.alertmanager.a.smithi122.stdout:Sep 11 08:08:31 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-alertmanager-a[73957]: ts=2024-09-11T08:08:31.315Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-09-11T08:08:31.735 INFO:journalctl@ceph.alertmanager.a.smithi122.stdout:Sep 11 08:08:31 smithi122 bash[73943]: bb573119c40ea6a5709763a232d56434b0fc14318b965be0a5f85aae27fb04ae 2024-09-11T08:08:31.735 INFO:journalctl@ceph.alertmanager.a.smithi122.stdout:Sep 11 08:08:31 smithi122 systemd[1]: Started Ceph alertmanager.a for f1d75850-7013-11ef-bcea-c7b262605968. 2024-09-11T08:08:32.271 INFO:journalctl@ceph.iscsi.iscsi.a.smithi135.stdout:Sep 11 08:08:31 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[59799]: debug there is no tcmu-runner data available 2024-09-11T08:08:32.677 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:32 smithi135 ceph-mon[32689]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 5.0 KiB/s rd, 426 B/s wr, 10 op/s; 48 B/s, 1 objects/s recovering 2024-09-11T08:08:32.677 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:32 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:32.677 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:32 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:32.677 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:32 smithi135 ceph-mon[32689]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-11T08:08:32.677 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:32 smithi135 ceph-mon[32689]: Reconfiguring daemon prometheus.a on smithi135 2024-09-11T08:08:32.677 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:32 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:08:32.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:32 smithi122 ceph-mon[39726]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 5.0 KiB/s rd, 426 B/s wr, 10 op/s; 48 B/s, 1 objects/s recovering 2024-09-11T08:08:32.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:32 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:32.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:32 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:32.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:32 smithi122 ceph-mon[39726]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-11T08:08:32.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:32 smithi122 ceph-mon[39726]: Reconfiguring daemon prometheus.a on smithi135 2024-09-11T08:08:32.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:32 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:08:32.735 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:32 smithi122 ceph-mon[28992]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 5.0 KiB/s rd, 426 B/s wr, 10 op/s; 48 B/s, 1 objects/s recovering 2024-09-11T08:08:32.735 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:32 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:32.735 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:32 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:32.735 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:32 smithi122 ceph-mon[28992]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-11T08:08:32.735 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:32 smithi122 ceph-mon[28992]: Reconfiguring daemon prometheus.a on smithi135 2024-09-11T08:08:32.735 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:32 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:08:33.638 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:33 smithi135 ceph-mon[32689]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:08:33.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:33 smithi122 ceph-mon[28992]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:08:33.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:33 smithi122 ceph-mon[39726]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:08:33.735 INFO:journalctl@ceph.alertmanager.a.smithi122.stdout:Sep 11 08:08:33 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-alertmanager-a[73957]: ts=2024-09-11T08:08:33.290Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.001086629s 2024-09-11T08:08:34.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:34 smithi135 ceph-mon[32689]: pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 5.4 KiB/s rd, 426 B/s wr, 10 op/s; 48 B/s, 1 objects/s recovering 2024-09-11T08:08:34.729 INFO:journalctl@ceph.prometheus.a.smithi135.stdout:Sep 11 08:08:34 smithi135 systemd[1]: Stopping Ceph prometheus.a for f1d75850-7013-11ef-bcea-c7b262605968... 2024-09-11T08:08:34.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:34 smithi122 ceph-mon[28992]: pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 5.4 KiB/s rd, 426 B/s wr, 10 op/s; 48 B/s, 1 objects/s recovering 2024-09-11T08:08:34.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:34 smithi122 ceph-mon[39726]: pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 5.4 KiB/s rd, 426 B/s wr, 10 op/s; 48 B/s, 1 objects/s recovering 2024-09-11T08:08:34.778 INFO:teuthology.orchestra.run.smithi122.stderr:++ ceph auth get-key osd.0 2024-09-11T08:08:35.054 INFO:journalctl@ceph.prometheus.a.smithi135.stdout:Sep 11 08:08:34 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-prometheus-a[61431]: ts=2024-09-11T08:08:34.745Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-09-11T08:08:35.055 INFO:journalctl@ceph.prometheus.a.smithi135.stdout:Sep 11 08:08:34 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-prometheus-a[61431]: ts=2024-09-11T08:08:34.745Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-09-11T08:08:35.055 INFO:journalctl@ceph.prometheus.a.smithi135.stdout:Sep 11 08:08:34 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-prometheus-a[61431]: ts=2024-09-11T08:08:34.745Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-09-11T08:08:35.055 INFO:journalctl@ceph.prometheus.a.smithi135.stdout:Sep 11 08:08:34 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-prometheus-a[61431]: ts=2024-09-11T08:08:34.745Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-09-11T08:08:35.055 INFO:journalctl@ceph.prometheus.a.smithi135.stdout:Sep 11 08:08:34 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-prometheus-a[61431]: ts=2024-09-11T08:08:34.745Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-09-11T08:08:35.055 INFO:journalctl@ceph.prometheus.a.smithi135.stdout:Sep 11 08:08:34 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-prometheus-a[61431]: ts=2024-09-11T08:08:34.745Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-09-11T08:08:35.055 INFO:journalctl@ceph.prometheus.a.smithi135.stdout:Sep 11 08:08:34 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-prometheus-a[61431]: ts=2024-09-11T08:08:34.745Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-09-11T08:08:35.055 INFO:journalctl@ceph.prometheus.a.smithi135.stdout:Sep 11 08:08:34 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-prometheus-a[61431]: ts=2024-09-11T08:08:34.745Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-09-11T08:08:35.055 INFO:journalctl@ceph.prometheus.a.smithi135.stdout:Sep 11 08:08:34 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-prometheus-a[61431]: ts=2024-09-11T08:08:34.745Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-09-11T08:08:35.055 INFO:journalctl@ceph.prometheus.a.smithi135.stdout:Sep 11 08:08:34 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-prometheus-a[61431]: ts=2024-09-11T08:08:34.745Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-09-11T08:08:35.055 INFO:journalctl@ceph.prometheus.a.smithi135.stdout:Sep 11 08:08:34 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-prometheus-a[61431]: ts=2024-09-11T08:08:34.745Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-09-11T08:08:35.055 INFO:journalctl@ceph.prometheus.a.smithi135.stdout:Sep 11 08:08:34 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-prometheus-a[61431]: ts=2024-09-11T08:08:34.745Z caller=main.go:1147 level=info msg="See you next time!" 2024-09-11T08:08:35.055 INFO:journalctl@ceph.prometheus.a.smithi135.stdout:Sep 11 08:08:34 smithi135 podman[64505]: 2024-09-11 08:08:34.755265108 +0000 UTC m=+0.144328814 container died efb99764bec219f38300f5bd756e4ef6fe0dd5926762943751747cf03c04188b (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T08:08:35.089 INFO:teuthology.orchestra.run.smithi122.stderr:+ NK=AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== 2024-09-11T08:08:35.090 INFO:teuthology.orchestra.run.smithi122.stderr:+ '[' AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== == AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== ']' 2024-09-11T08:08:35.090 INFO:teuthology.orchestra.run.smithi122.stderr:+ sleep 5 2024-09-11T08:08:35.371 INFO:journalctl@ceph.prometheus.a.smithi135.stdout:Sep 11 08:08:35 smithi135 podman[64505]: 2024-09-11 08:08:35.169829661 +0000 UTC m=+0.558893364 container cleanup efb99764bec219f38300f5bd756e4ef6fe0dd5926762943751747cf03c04188b (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T08:08:35.371 INFO:journalctl@ceph.prometheus.a.smithi135.stdout:Sep 11 08:08:35 smithi135 bash[64505]: ceph-f1d75850-7013-11ef-bcea-c7b262605968-prometheus-a 2024-09-11T08:08:35.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:35 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/791227000' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:08:35.728 INFO:journalctl@ceph.prometheus.a.smithi135.stdout:Sep 11 08:08:35 smithi135 podman[64519]: 2024-09-11 08:08:35.370969201 +0000 UTC m=+0.617496790 container remove efb99764bec219f38300f5bd756e4ef6fe0dd5926762943751747cf03c04188b (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T08:08:35.728 INFO:journalctl@ceph.prometheus.a.smithi135.stdout:Sep 11 08:08:35 smithi135 systemd[1]: ceph-f1d75850-7013-11ef-bcea-c7b262605968@prometheus.a.service: Deactivated successfully. 2024-09-11T08:08:35.728 INFO:journalctl@ceph.prometheus.a.smithi135.stdout:Sep 11 08:08:35 smithi135 systemd[1]: Stopped Ceph prometheus.a for f1d75850-7013-11ef-bcea-c7b262605968. 2024-09-11T08:08:35.728 INFO:journalctl@ceph.prometheus.a.smithi135.stdout:Sep 11 08:08:35 smithi135 systemd[1]: ceph-f1d75850-7013-11ef-bcea-c7b262605968@prometheus.a.service: Consumed 1.031s CPU time. 2024-09-11T08:08:35.729 INFO:journalctl@ceph.prometheus.a.smithi135.stdout:Sep 11 08:08:35 smithi135 systemd[1]: Starting Ceph prometheus.a for f1d75850-7013-11ef-bcea-c7b262605968... 2024-09-11T08:08:35.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:35 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/791227000' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:08:35.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:35 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/791227000' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:08:36.052 INFO:journalctl@ceph.prometheus.a.smithi135.stdout:Sep 11 08:08:35 smithi135 podman[64609]: 2024-09-11 08:08:35.703907458 +0000 UTC m=+0.016968745 image pull a07b618ecd1dce142bce4c52f0e80982eaf1f14265a2415c2d35978ccaa0a464 quay.io/prometheus/prometheus:v2.43.0 2024-09-11T08:08:36.052 INFO:journalctl@ceph.prometheus.a.smithi135.stdout:Sep 11 08:08:35 smithi135 podman[64609]: 2024-09-11 08:08:35.838758353 +0000 UTC m=+0.151819631 container create faf0895c4ca57d977ae69ff6f4c21fa41b6054eda0ba470e7d049d47bc802118 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T08:08:36.052 INFO:journalctl@ceph.prometheus.a.smithi135.stdout:Sep 11 08:08:36 smithi135 podman[64609]: 2024-09-11 08:08:36.049256311 +0000 UTC m=+0.362317607 container init faf0895c4ca57d977ae69ff6f4c21fa41b6054eda0ba470e7d049d47bc802118 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T08:08:36.412 INFO:journalctl@ceph.prometheus.a.smithi135.stdout:Sep 11 08:08:36 smithi135 podman[64609]: 2024-09-11 08:08:36.051975769 +0000 UTC m=+0.365037059 container start faf0895c4ca57d977ae69ff6f4c21fa41b6054eda0ba470e7d049d47bc802118 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T08:08:36.413 INFO:journalctl@ceph.prometheus.a.smithi135.stdout:Sep 11 08:08:36 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-prometheus-a[64624]: ts=2024-09-11T08:08:36.075Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2024-09-11T08:08:36.413 INFO:journalctl@ceph.prometheus.a.smithi135.stdout:Sep 11 08:08:36 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-prometheus-a[64624]: ts=2024-09-11T08:08:36.075Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2024-09-11T08:08:36.413 INFO:journalctl@ceph.prometheus.a.smithi135.stdout:Sep 11 08:08:36 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-prometheus-a[64624]: ts=2024-09-11T08:08:36.075Z caller=main.go:570 level=info host_details="(Linux 5.14.0-505.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Sep 5 07:54:07 UTC 2024 x86_64 smithi135 (none))" 2024-09-11T08:08:36.413 INFO:journalctl@ceph.prometheus.a.smithi135.stdout:Sep 11 08:08:36 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-prometheus-a[64624]: ts=2024-09-11T08:08:36.075Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-09-11T08:08:36.413 INFO:journalctl@ceph.prometheus.a.smithi135.stdout:Sep 11 08:08:36 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-prometheus-a[64624]: ts=2024-09-11T08:08:36.075Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-09-11T08:08:36.413 INFO:journalctl@ceph.prometheus.a.smithi135.stdout:Sep 11 08:08:36 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-prometheus-a[64624]: ts=2024-09-11T08:08:36.077Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2024-09-11T08:08:36.413 INFO:journalctl@ceph.prometheus.a.smithi135.stdout:Sep 11 08:08:36 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-prometheus-a[64624]: ts=2024-09-11T08:08:36.077Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2024-09-11T08:08:36.413 INFO:journalctl@ceph.prometheus.a.smithi135.stdout:Sep 11 08:08:36 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-prometheus-a[64624]: ts=2024-09-11T08:08:36.078Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2024-09-11T08:08:36.413 INFO:journalctl@ceph.prometheus.a.smithi135.stdout:Sep 11 08:08:36 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-prometheus-a[64624]: ts=2024-09-11T08:08:36.078Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-09-11T08:08:36.413 INFO:journalctl@ceph.prometheus.a.smithi135.stdout:Sep 11 08:08:36 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-prometheus-a[64624]: ts=2024-09-11T08:08:36.079Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-09-11T08:08:36.413 INFO:journalctl@ceph.prometheus.a.smithi135.stdout:Sep 11 08:08:36 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-prometheus-a[64624]: ts=2024-09-11T08:08:36.079Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=1.629µs 2024-09-11T08:08:36.414 INFO:journalctl@ceph.prometheus.a.smithi135.stdout:Sep 11 08:08:36 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-prometheus-a[64624]: ts=2024-09-11T08:08:36.079Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-09-11T08:08:36.414 INFO:journalctl@ceph.prometheus.a.smithi135.stdout:Sep 11 08:08:36 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-prometheus-a[64624]: ts=2024-09-11T08:08:36.080Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=1 2024-09-11T08:08:36.414 INFO:journalctl@ceph.prometheus.a.smithi135.stdout:Sep 11 08:08:36 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-prometheus-a[64624]: ts=2024-09-11T08:08:36.081Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=1 maxSegment=1 2024-09-11T08:08:36.414 INFO:journalctl@ceph.prometheus.a.smithi135.stdout:Sep 11 08:08:36 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-prometheus-a[64624]: ts=2024-09-11T08:08:36.081Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=35.707µs wal_replay_duration=1.370168ms wbl_replay_duration=146ns total_replay_duration=1.428818ms 2024-09-11T08:08:36.414 INFO:journalctl@ceph.prometheus.a.smithi135.stdout:Sep 11 08:08:36 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-prometheus-a[64624]: ts=2024-09-11T08:08:36.082Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2024-09-11T08:08:36.414 INFO:journalctl@ceph.prometheus.a.smithi135.stdout:Sep 11 08:08:36 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-prometheus-a[64624]: ts=2024-09-11T08:08:36.082Z caller=main.go:1029 level=info msg="TSDB started" 2024-09-11T08:08:36.414 INFO:journalctl@ceph.prometheus.a.smithi135.stdout:Sep 11 08:08:36 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-prometheus-a[64624]: ts=2024-09-11T08:08:36.082Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-09-11T08:08:36.414 INFO:journalctl@ceph.prometheus.a.smithi135.stdout:Sep 11 08:08:36 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-prometheus-a[64624]: ts=2024-09-11T08:08:36.096Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=14.355555ms db_storage=796ns remote_storage=1.41µs web_handler=365ns query_engine=437ns scrape=424.639µs scrape_sd=27.171µs notify=18.461µs notify_sd=5.331µs rules=13.6399ms tracing=9.158µs 2024-09-11T08:08:36.414 INFO:journalctl@ceph.prometheus.a.smithi135.stdout:Sep 11 08:08:36 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-prometheus-a[64624]: ts=2024-09-11T08:08:36.096Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-09-11T08:08:36.414 INFO:journalctl@ceph.prometheus.a.smithi135.stdout:Sep 11 08:08:36 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-prometheus-a[64624]: ts=2024-09-11T08:08:36.096Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-09-11T08:08:36.414 INFO:journalctl@ceph.prometheus.a.smithi135.stdout:Sep 11 08:08:36 smithi135 bash[64609]: faf0895c4ca57d977ae69ff6f4c21fa41b6054eda0ba470e7d049d47bc802118 2024-09-11T08:08:36.414 INFO:journalctl@ceph.prometheus.a.smithi135.stdout:Sep 11 08:08:36 smithi135 systemd[1]: Started Ceph prometheus.a for f1d75850-7013-11ef-bcea-c7b262605968. 2024-09-11T08:08:36.484 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:08:36 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: [11/Sep/2024:08:08:36] ENGINE Bus STOPPING 2024-09-11T08:08:36.484 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:08:36 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: [11/Sep/2024:08:08:36] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-11T08:08:36.484 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:08:36 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: [11/Sep/2024:08:08:36] ENGINE Bus STOPPED 2024-09-11T08:08:36.484 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:08:36 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: [11/Sep/2024:08:08:36] ENGINE Bus STARTING 2024-09-11T08:08:36.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:36 smithi122 ceph-mon[39726]: pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:08:36.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:36 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:36.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:36 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:36.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:36 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-11T08:08:36.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:36 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi122.front.sepia.ceph.com:9093"}]: dispatch 2024-09-11T08:08:36.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:36 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:36.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:36 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-11T08:08:36.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:36 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi135.front.sepia.ceph.com:9095"}]: dispatch 2024-09-11T08:08:36.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:36 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:36.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:36 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-11T08:08:36.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:36 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi135.front.sepia.ceph.com:3000"}]: dispatch 2024-09-11T08:08:36.486 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:36 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:36.486 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:36 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:08:36.486 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:36 smithi122 ceph-mon[28992]: pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:08:36.486 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:36 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:36.486 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:36 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:36.486 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:36 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-11T08:08:36.486 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:36 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi122.front.sepia.ceph.com:9093"}]: dispatch 2024-09-11T08:08:36.486 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:36 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:36.486 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:36 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-11T08:08:36.486 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:36 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi135.front.sepia.ceph.com:9095"}]: dispatch 2024-09-11T08:08:36.486 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:36 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:36.487 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:36 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-11T08:08:36.487 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:36 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi135.front.sepia.ceph.com:3000"}]: dispatch 2024-09-11T08:08:36.487 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:36 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:36.487 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:36 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:08:36.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:36 smithi135 ceph-mon[32689]: pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:08:36.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:36 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:36.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:36 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:36.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:36 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-11T08:08:36.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:36 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi122.front.sepia.ceph.com:9093"}]: dispatch 2024-09-11T08:08:36.729 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:36 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:36.729 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:36 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-11T08:08:36.729 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:36 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi135.front.sepia.ceph.com:9095"}]: dispatch 2024-09-11T08:08:36.729 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:36 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:36.729 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:36 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-11T08:08:36.729 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:36 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi135.front.sepia.ceph.com:3000"}]: dispatch 2024-09-11T08:08:36.729 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:36 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:36.729 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:36 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:08:36.893 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:08:36 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: [11/Sep/2024:08:08:36] ENGINE Serving on http://:::9283 2024-09-11T08:08:36.894 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:08:36 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: [11/Sep/2024:08:08:36] ENGINE Bus STARTED 2024-09-11T08:08:36.894 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:08:36 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: [11/Sep/2024:08:08:36] ENGINE Bus STOPPING 2024-09-11T08:08:37.147 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:08:37 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: [11/Sep/2024:08:08:37] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-11T08:08:37.148 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:08:37 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: [11/Sep/2024:08:08:37] ENGINE Bus STOPPED 2024-09-11T08:08:37.148 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:08:37 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: [11/Sep/2024:08:08:37] ENGINE Bus STARTING 2024-09-11T08:08:37.148 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:08:37 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: [11/Sep/2024:08:08:37] ENGINE Serving on http://:::9283 2024-09-11T08:08:37.417 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:08:37 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: [11/Sep/2024:08:08:37] ENGINE Bus STARTED 2024-09-11T08:08:37.418 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:08:37 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: [11/Sep/2024:08:08:37] ENGINE Bus STOPPING 2024-09-11T08:08:37.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:37 smithi135 ceph-mon[32689]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-11T08:08:37.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:37 smithi135 ceph-mon[32689]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi122.front.sepia.ceph.com:9093"}]: dispatch 2024-09-11T08:08:37.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:37 smithi135 ceph-mon[32689]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-11T08:08:37.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:37 smithi135 ceph-mon[32689]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi135.front.sepia.ceph.com:9095"}]: dispatch 2024-09-11T08:08:37.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:37 smithi135 ceph-mon[32689]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-11T08:08:37.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:37 smithi135 ceph-mon[32689]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi135.front.sepia.ceph.com:3000"}]: dispatch 2024-09-11T08:08:37.734 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:08:37 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: [11/Sep/2024:08:08:37] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-11T08:08:37.734 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:08:37 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: [11/Sep/2024:08:08:37] ENGINE Bus STOPPED 2024-09-11T08:08:37.734 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:08:37 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: [11/Sep/2024:08:08:37] ENGINE Bus STARTING 2024-09-11T08:08:37.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:37 smithi122 ceph-mon[28992]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-11T08:08:37.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:37 smithi122 ceph-mon[28992]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi122.front.sepia.ceph.com:9093"}]: dispatch 2024-09-11T08:08:37.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:37 smithi122 ceph-mon[28992]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-11T08:08:37.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:37 smithi122 ceph-mon[28992]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi135.front.sepia.ceph.com:9095"}]: dispatch 2024-09-11T08:08:37.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:37 smithi122 ceph-mon[28992]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-11T08:08:37.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:37 smithi122 ceph-mon[28992]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi135.front.sepia.ceph.com:3000"}]: dispatch 2024-09-11T08:08:37.735 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:37 smithi122 ceph-mon[39726]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-11T08:08:37.735 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:37 smithi122 ceph-mon[39726]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi122.front.sepia.ceph.com:9093"}]: dispatch 2024-09-11T08:08:37.735 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:37 smithi122 ceph-mon[39726]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-11T08:08:37.735 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:37 smithi122 ceph-mon[39726]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi135.front.sepia.ceph.com:9095"}]: dispatch 2024-09-11T08:08:37.735 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:37 smithi122 ceph-mon[39726]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-11T08:08:37.735 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:37 smithi122 ceph-mon[39726]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi135.front.sepia.ceph.com:3000"}]: dispatch 2024-09-11T08:08:38.234 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:08:37 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: [11/Sep/2024:08:08:37] ENGINE Serving on http://:::9283 2024-09-11T08:08:38.234 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:08:37 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: [11/Sep/2024:08:08:37] ENGINE Bus STARTED 2024-09-11T08:08:38.707 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:38 smithi122 ceph-mon[39726]: pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:08:38.708 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:38 smithi122 ceph-mon[28992]: pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:08:38.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:38 smithi135 ceph-mon[32689]: pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:08:40.092 INFO:teuthology.orchestra.run.smithi122.stderr:++ ceph auth get-key osd.0 2024-09-11T08:08:40.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:39 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:40.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:39 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:40.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:39 smithi135 ceph-mon[32689]: pgmap v46: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:08:40.234 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:39 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:40.234 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:39 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:40.234 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:39 smithi122 ceph-mon[39726]: pgmap v46: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:08:40.234 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:39 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:40.234 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:39 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:40.234 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:39 smithi122 ceph-mon[28992]: pgmap v46: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:08:40.397 INFO:teuthology.orchestra.run.smithi122.stderr:+ NK=AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== 2024-09-11T08:08:40.397 INFO:teuthology.orchestra.run.smithi122.stderr:+ '[' AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== == AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== ']' 2024-09-11T08:08:40.397 INFO:teuthology.orchestra.run.smithi122.stderr:+ sleep 5 2024-09-11T08:08:41.208 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:40 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/1935406800' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:08:41.234 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:40 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/1935406800' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:08:41.234 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:40 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/1935406800' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:08:41.734 INFO:journalctl@ceph.alertmanager.a.smithi122.stdout:Sep 11 08:08:41 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-alertmanager-a[73957]: ts=2024-09-11T08:08:41.293Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.004483936s 2024-09-11T08:08:42.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:41 smithi135 ceph-mon[32689]: pgmap v47: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:08:42.165 INFO:journalctl@ceph.iscsi.iscsi.a.smithi135.stdout:Sep 11 08:08:41 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[59799]: debug there is no tcmu-runner data available 2024-09-11T08:08:42.234 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:41 smithi122 ceph-mon[28992]: pgmap v47: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:08:42.234 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:41 smithi122 ceph-mon[39726]: pgmap v47: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:08:43.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:42 smithi135 ceph-mon[32689]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:08:43.234 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:42 smithi122 ceph-mon[28992]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:08:43.234 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:42 smithi122 ceph-mon[39726]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:08:43.609 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:08:43 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: ::ffff:172.21.15.135 - - [11/Sep/2024:08:08:43] "GET /metrics HTTP/1.1" 200 219526 "" "Prometheus/2.43.0" 2024-09-11T08:08:44.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:43 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:44.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:43 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:44.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:43 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:08:44.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:43 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:08:44.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:43 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:44.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:43 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:08:44.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:43 smithi135 ceph-mon[32689]: pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:08:44.234 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:43 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:44.234 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:43 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:44.234 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:43 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:08:44.234 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:43 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:08:44.234 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:43 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:44.234 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:43 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:08:44.235 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:43 smithi122 ceph-mon[28992]: pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:08:44.235 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:43 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:44.235 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:43 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:44.235 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:43 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:08:44.235 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:43 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:08:44.235 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:43 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:44.235 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:43 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:08:44.236 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:43 smithi122 ceph-mon[39726]: pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:08:45.399 INFO:teuthology.orchestra.run.smithi122.stderr:++ ceph auth get-key osd.0 2024-09-11T08:08:45.705 INFO:teuthology.orchestra.run.smithi122.stderr:+ NK=AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== 2024-09-11T08:08:45.705 INFO:teuthology.orchestra.run.smithi122.stderr:+ '[' AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== == AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== ']' 2024-09-11T08:08:45.705 INFO:teuthology.orchestra.run.smithi122.stderr:+ sleep 5 2024-09-11T08:08:46.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:46 smithi135 ceph-mon[32689]: pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:08:46.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:46 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/183049405' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:08:46.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:46 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:46.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:46 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:46.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:46 smithi122 ceph-mon[28992]: pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:08:46.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:46 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/183049405' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:08:46.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:46 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:46.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:46 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:46.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:46 smithi122 ceph-mon[39726]: pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:08:46.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:46 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/183049405' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:08:46.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:46 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:46.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:46 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:47.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:47 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:08:47.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:47 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:08:47.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:47 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:08:48.228 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:08:47 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: ::ffff:172.21.15.135 - - [11/Sep/2024:08:08:47] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:08:48.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:48 smithi135 ceph-mon[32689]: pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:08:48.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:48 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:48.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:48 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:48.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:48 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:08:48.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:48 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:08:48.229 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:48 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:48.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:48 smithi122 ceph-mon[28992]: pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:08:48.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:48 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:48.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:48 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:48.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:48 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:08:48.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:48 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:08:48.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:48 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:48.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:48 smithi122 ceph-mon[39726]: pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:08:48.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:48 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:48.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:48 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:48.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:48 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:08:48.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:48 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:08:48.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:48 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:08:50.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:50 smithi135 ceph-mon[32689]: pgmap v51: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:08:50.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:50 smithi122 ceph-mon[39726]: pgmap v51: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:08:50.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:50 smithi122 ceph-mon[28992]: pgmap v51: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:08:50.707 INFO:teuthology.orchestra.run.smithi122.stderr:++ ceph auth get-key osd.0 2024-09-11T08:08:51.018 INFO:teuthology.orchestra.run.smithi122.stderr:+ NK=AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== 2024-09-11T08:08:51.019 INFO:teuthology.orchestra.run.smithi122.stderr:+ '[' AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== == AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== ']' 2024-09-11T08:08:51.019 INFO:teuthology.orchestra.run.smithi122.stderr:+ sleep 5 2024-09-11T08:08:51.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:51 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/1486050681' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:08:51.483 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:51 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/1486050681' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:08:51.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:51 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/1486050681' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:08:52.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:52 smithi135 ceph-mon[32689]: pgmap v52: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:08:52.478 INFO:journalctl@ceph.iscsi.iscsi.a.smithi135.stdout:Sep 11 08:08:52 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[59799]: debug there is no tcmu-runner data available 2024-09-11T08:08:52.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:52 smithi122 ceph-mon[39726]: pgmap v52: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:08:52.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:52 smithi122 ceph-mon[28992]: pgmap v52: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:08:53.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:53 smithi135 ceph-mon[32689]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:08:53.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:53 smithi122 ceph-mon[28992]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:08:53.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:53 smithi122 ceph-mon[39726]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:08:53.484 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:08:53 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: ::ffff:172.21.15.135 - - [11/Sep/2024:08:08:53] "GET /metrics HTTP/1.1" 200 219521 "" "Prometheus/2.43.0" 2024-09-11T08:08:54.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:54 smithi135 ceph-mon[32689]: pgmap v53: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:08:54.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:54 smithi122 ceph-mon[39726]: pgmap v53: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:08:54.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:54 smithi122 ceph-mon[28992]: pgmap v53: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:08:56.021 INFO:teuthology.orchestra.run.smithi122.stderr:++ ceph auth get-key osd.0 2024-09-11T08:08:56.338 INFO:teuthology.orchestra.run.smithi122.stderr:+ NK=AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== 2024-09-11T08:08:56.338 INFO:teuthology.orchestra.run.smithi122.stderr:+ '[' AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== == AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== ']' 2024-09-11T08:08:56.338 INFO:teuthology.orchestra.run.smithi122.stderr:+ sleep 5 2024-09-11T08:08:56.338 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:56 smithi122 ceph-mon[28992]: pgmap v54: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:08:56.339 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:56 smithi122 ceph-mon[39726]: pgmap v54: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:08:56.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:56 smithi135 ceph-mon[32689]: pgmap v54: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:08:57.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:57 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/803199198' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:08:57.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:57 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/803199198' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:08:57.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:57 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/803199198' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:08:58.202 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:08:57 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: ::ffff:172.21.15.135 - - [11/Sep/2024:08:08:57] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:08:58.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:08:58 smithi135 ceph-mon[32689]: pgmap v55: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:08:58.483 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:08:58 smithi122 ceph-mon[28992]: pgmap v55: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:08:58.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:08:58 smithi122 ceph-mon[39726]: pgmap v55: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:09:00.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:09:00 smithi135 ceph-mon[32689]: pgmap v56: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:09:00.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:09:00 smithi122 ceph-mon[28992]: pgmap v56: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:09:00.486 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:09:00 smithi122 ceph-mon[39726]: pgmap v56: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:09:01.340 INFO:teuthology.orchestra.run.smithi122.stderr:++ ceph auth get-key osd.0 2024-09-11T08:09:01.650 INFO:teuthology.orchestra.run.smithi122.stderr:+ NK=AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== 2024-09-11T08:09:01.650 INFO:teuthology.orchestra.run.smithi122.stderr:+ '[' AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== == AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== ']' 2024-09-11T08:09:01.650 INFO:teuthology.orchestra.run.smithi122.stderr:+ sleep 5 2024-09-11T08:09:02.478 INFO:journalctl@ceph.iscsi.iscsi.a.smithi135.stdout:Sep 11 08:09:02 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[59799]: debug there is no tcmu-runner data available 2024-09-11T08:09:02.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:09:02 smithi135 ceph-mon[32689]: pgmap v57: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:09:02.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:09:02 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/3453478750' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:09:02.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:09:02 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:09:02.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:09:02 smithi122 ceph-mon[39726]: pgmap v57: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:09:02.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:09:02 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/3453478750' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:09:02.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:09:02 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:09:02.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:09:02 smithi122 ceph-mon[28992]: pgmap v57: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:09:02.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:09:02 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/3453478750' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:09:02.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:09:02 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:09:03.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:09:03 smithi135 ceph-mon[32689]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:09:03.483 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:09:03 smithi122 ceph-mon[39726]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:09:03.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:09:03 smithi122 ceph-mon[28992]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:09:03.484 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:09:03 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: ::ffff:172.21.15.135 - - [11/Sep/2024:08:09:03] "GET /metrics HTTP/1.1" 200 219492 "" "Prometheus/2.43.0" 2024-09-11T08:09:04.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:09:04 smithi135 ceph-mon[32689]: pgmap v58: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:09:04.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:09:04 smithi122 ceph-mon[39726]: pgmap v58: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:09:04.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:09:04 smithi122 ceph-mon[28992]: pgmap v58: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:09:06.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:09:06 smithi135 ceph-mon[32689]: pgmap v59: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:09:06.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:09:06 smithi122 ceph-mon[39726]: pgmap v59: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:09:06.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:09:06 smithi122 ceph-mon[28992]: pgmap v59: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:09:06.652 INFO:teuthology.orchestra.run.smithi122.stderr:++ ceph auth get-key osd.0 2024-09-11T08:09:06.960 INFO:teuthology.orchestra.run.smithi122.stderr:+ NK=AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== 2024-09-11T08:09:06.960 INFO:teuthology.orchestra.run.smithi122.stderr:+ '[' AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== == AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== ']' 2024-09-11T08:09:06.961 INFO:teuthology.orchestra.run.smithi122.stderr:+ sleep 5 2024-09-11T08:09:07.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:09:07 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/762480571' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:09:07.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:09:07 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/762480571' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:09:07.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:09:07 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/762480571' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:09:08.228 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:09:07 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: ::ffff:172.21.15.135 - - [11/Sep/2024:08:09:07] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:09:08.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:09:08 smithi122 ceph-mon[28992]: pgmap v60: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:09:08.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:09:08 smithi122 ceph-mon[39726]: pgmap v60: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:09:08.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:09:08 smithi135 ceph-mon[32689]: pgmap v60: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:09:10.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:09:10 smithi122 ceph-mon[39726]: pgmap v61: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:09:10.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:09:10 smithi122 ceph-mon[28992]: pgmap v61: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:09:10.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:09:10 smithi135 ceph-mon[32689]: pgmap v61: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:09:11.962 INFO:teuthology.orchestra.run.smithi122.stderr:++ ceph auth get-key osd.0 2024-09-11T08:09:12.281 INFO:teuthology.orchestra.run.smithi122.stderr:+ NK=AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== 2024-09-11T08:09:12.281 INFO:teuthology.orchestra.run.smithi122.stderr:+ '[' AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== == AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== ']' 2024-09-11T08:09:12.281 INFO:teuthology.orchestra.run.smithi122.stderr:+ sleep 5 2024-09-11T08:09:12.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:09:12 smithi135 ceph-mon[32689]: pgmap v62: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:09:12.478 INFO:journalctl@ceph.iscsi.iscsi.a.smithi135.stdout:Sep 11 08:09:12 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[59799]: debug there is no tcmu-runner data available 2024-09-11T08:09:12.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:09:12 smithi122 ceph-mon[28992]: pgmap v62: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:09:12.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:09:12 smithi122 ceph-mon[39726]: pgmap v62: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:09:13.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:09:13 smithi135 ceph-mon[32689]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:09:13.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:09:13 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/1727583987' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:09:13.734 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:09:13 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: ::ffff:172.21.15.135 - - [11/Sep/2024:08:09:13] "GET /metrics HTTP/1.1" 200 219492 "" "Prometheus/2.43.0" 2024-09-11T08:09:13.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:09:13 smithi122 ceph-mon[28992]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:09:13.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:09:13 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/1727583987' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:09:13.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:09:13 smithi122 ceph-mon[39726]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:09:13.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:09:13 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/1727583987' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:09:14.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:09:14 smithi135 ceph-mon[32689]: pgmap v63: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:09:14.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:09:14 smithi122 ceph-mon[28992]: pgmap v63: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:09:14.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:09:14 smithi122 ceph-mon[39726]: pgmap v63: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:09:16.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:09:16 smithi135 ceph-mon[32689]: pgmap v64: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:09:16.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:09:16 smithi122 ceph-mon[39726]: pgmap v64: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:09:16.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:09:16 smithi122 ceph-mon[28992]: pgmap v64: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:09:17.282 INFO:teuthology.orchestra.run.smithi122.stderr:++ ceph auth get-key osd.0 2024-09-11T08:09:17.596 INFO:teuthology.orchestra.run.smithi122.stderr:+ NK=AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== 2024-09-11T08:09:17.597 INFO:teuthology.orchestra.run.smithi122.stderr:+ '[' AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== == AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== ']' 2024-09-11T08:09:17.597 INFO:teuthology.orchestra.run.smithi122.stderr:+ sleep 5 2024-09-11T08:09:17.597 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:09:17 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:09:17.597 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:09:17 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:09:17.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:09:17 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:09:18.228 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:09:17 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: ::ffff:172.21.15.135 - - [11/Sep/2024:08:09:17] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:09:18.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:09:18 smithi135 ceph-mon[32689]: pgmap v65: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:09:18.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:09:18 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/3505039114' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:09:18.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:09:18 smithi122 ceph-mon[39726]: pgmap v65: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:09:18.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:09:18 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/3505039114' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:09:18.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:09:18 smithi122 ceph-mon[28992]: pgmap v65: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:09:18.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:09:18 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/3505039114' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:09:20.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:09:20 smithi135 ceph-mon[32689]: pgmap v66: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:09:20.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:09:20 smithi122 ceph-mon[28992]: pgmap v66: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:09:20.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:09:20 smithi122 ceph-mon[39726]: pgmap v66: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:09:22.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:09:22 smithi135 ceph-mon[32689]: pgmap v67: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:09:22.478 INFO:journalctl@ceph.iscsi.iscsi.a.smithi135.stdout:Sep 11 08:09:22 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[59799]: debug there is no tcmu-runner data available 2024-09-11T08:09:22.598 INFO:teuthology.orchestra.run.smithi122.stderr:++ ceph auth get-key osd.0 2024-09-11T08:09:22.599 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:09:22 smithi122 ceph-mon[28992]: pgmap v67: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:09:22.599 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:09:22 smithi122 ceph-mon[39726]: pgmap v67: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:09:22.909 INFO:teuthology.orchestra.run.smithi122.stderr:+ NK=AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== 2024-09-11T08:09:22.909 INFO:teuthology.orchestra.run.smithi122.stderr:+ '[' AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== == AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== ']' 2024-09-11T08:09:22.910 INFO:teuthology.orchestra.run.smithi122.stderr:+ sleep 5 2024-09-11T08:09:23.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:09:23 smithi135 ceph-mon[32689]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:09:23.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:09:23 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/17541931' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:09:23.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:09:23 smithi122 ceph-mon[28992]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:09:23.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:09:23 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/17541931' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:09:23.734 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:09:23 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: ::ffff:172.21.15.135 - - [11/Sep/2024:08:09:23] "GET /metrics HTTP/1.1" 200 219499 "" "Prometheus/2.43.0" 2024-09-11T08:09:23.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:09:23 smithi122 ceph-mon[39726]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:09:23.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:09:23 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/17541931' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:09:24.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:09:24 smithi135 ceph-mon[32689]: pgmap v68: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:09:24.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:09:24 smithi122 ceph-mon[28992]: pgmap v68: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:09:24.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:09:24 smithi122 ceph-mon[39726]: pgmap v68: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:09:26.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:09:26 smithi135 ceph-mon[32689]: pgmap v69: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:09:26.733 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:09:26 smithi122 ceph-mon[28992]: pgmap v69: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:09:26.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:09:26 smithi122 ceph-mon[39726]: pgmap v69: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:09:27.911 INFO:teuthology.orchestra.run.smithi122.stderr:++ ceph auth get-key osd.0 2024-09-11T08:09:28.225 INFO:teuthology.orchestra.run.smithi122.stderr:+ NK=AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== 2024-09-11T08:09:28.225 INFO:teuthology.orchestra.run.smithi122.stderr:+ '[' AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== == AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== ']' 2024-09-11T08:09:28.225 INFO:teuthology.orchestra.run.smithi122.stderr:+ sleep 5 2024-09-11T08:09:28.228 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:09:27 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: ::ffff:172.21.15.135 - - [11/Sep/2024:08:09:27] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:09:28.476 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:09:28 smithi122 ceph-mon[28992]: pgmap v70: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:09:28.476 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:09:28 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/1018064021' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:09:28.477 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:09:28 smithi122 ceph-mon[39726]: pgmap v70: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:09:28.477 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:09:28 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/1018064021' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:09:28.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:09:28 smithi135 ceph-mon[32689]: pgmap v70: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:09:28.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:09:28 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/1018064021' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:09:30.531 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:09:30 smithi135 ceph-mon[32689]: pgmap v71: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:09:30.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:09:30 smithi122 ceph-mon[28992]: pgmap v71: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:09:30.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:09:30 smithi122 ceph-mon[39726]: pgmap v71: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:09:32.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:09:32 smithi135 ceph-mon[32689]: pgmap v72: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:09:32.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:09:32 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:09:32.478 INFO:journalctl@ceph.iscsi.iscsi.a.smithi135.stdout:Sep 11 08:09:32 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[59799]: debug there is no tcmu-runner data available 2024-09-11T08:09:32.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:09:32 smithi122 ceph-mon[28992]: pgmap v72: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:09:32.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:09:32 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:09:32.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:09:32 smithi122 ceph-mon[39726]: pgmap v72: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:09:32.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:09:32 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:09:33.227 INFO:teuthology.orchestra.run.smithi122.stderr:++ ceph auth get-key osd.0 2024-09-11T08:09:33.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:09:33 smithi122 ceph-mon[28992]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:09:33.484 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:09:33 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: ::ffff:172.21.15.135 - - [11/Sep/2024:08:09:33] "GET /metrics HTTP/1.1" 200 219508 "" "Prometheus/2.43.0" 2024-09-11T08:09:33.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:09:33 smithi122 ceph-mon[39726]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:09:33.543 INFO:teuthology.orchestra.run.smithi122.stderr:+ NK=AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== 2024-09-11T08:09:33.543 INFO:teuthology.orchestra.run.smithi122.stderr:+ '[' AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== == AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== ']' 2024-09-11T08:09:33.543 INFO:teuthology.orchestra.run.smithi122.stderr:+ sleep 5 2024-09-11T08:09:33.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:09:33 smithi135 ceph-mon[32689]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:09:34.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:09:34 smithi135 ceph-mon[32689]: pgmap v73: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:09:34.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:09:34 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/336172015' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:09:34.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:09:34 smithi122 ceph-mon[28992]: pgmap v73: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:09:34.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:09:34 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/336172015' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:09:34.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:09:34 smithi122 ceph-mon[39726]: pgmap v73: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:09:34.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:09:34 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/336172015' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:09:36.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:09:36 smithi135 ceph-mon[32689]: pgmap v74: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:09:36.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:09:36 smithi122 ceph-mon[28992]: pgmap v74: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:09:36.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:09:36 smithi122 ceph-mon[39726]: pgmap v74: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:09:38.228 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:09:37 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: ::ffff:172.21.15.135 - - [11/Sep/2024:08:09:37] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:09:38.545 INFO:teuthology.orchestra.run.smithi122.stderr:++ ceph auth get-key osd.0 2024-09-11T08:09:38.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:09:38 smithi135 ceph-mon[32689]: pgmap v75: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:09:38.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:09:38 smithi122 ceph-mon[28992]: pgmap v75: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:09:38.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:09:38 smithi122 ceph-mon[39726]: pgmap v75: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:09:38.853 INFO:teuthology.orchestra.run.smithi122.stderr:+ NK=AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== 2024-09-11T08:09:38.853 INFO:teuthology.orchestra.run.smithi122.stderr:+ '[' AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== == AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== ']' 2024-09-11T08:09:38.853 INFO:teuthology.orchestra.run.smithi122.stderr:+ sleep 5 2024-09-11T08:09:39.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:09:39 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/3180813362' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:09:39.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:09:39 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/3180813362' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:09:39.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:09:39 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/3180813362' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:09:40.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:09:40 smithi135 ceph-mon[32689]: pgmap v76: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:09:40.733 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:09:40 smithi122 ceph-mon[28992]: pgmap v76: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:09:40.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:09:40 smithi122 ceph-mon[39726]: pgmap v76: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:09:42.401 INFO:journalctl@ceph.iscsi.iscsi.a.smithi135.stdout:Sep 11 08:09:42 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[59799]: debug there is no tcmu-runner data available 2024-09-11T08:09:42.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:09:42 smithi135 ceph-mon[32689]: pgmap v77: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:09:42.733 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:09:42 smithi122 ceph-mon[28992]: pgmap v77: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:09:42.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:09:42 smithi122 ceph-mon[39726]: pgmap v77: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:09:43.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:09:43 smithi135 ceph-mon[32689]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:09:43.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:09:43 smithi122 ceph-mon[39726]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:09:43.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:09:43 smithi122 ceph-mon[28992]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:09:43.734 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:09:43 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: ::ffff:172.21.15.135 - - [11/Sep/2024:08:09:43] "GET /metrics HTTP/1.1" 200 219508 "" "Prometheus/2.43.0" 2024-09-11T08:09:43.855 INFO:teuthology.orchestra.run.smithi122.stderr:++ ceph auth get-key osd.0 2024-09-11T08:09:44.163 INFO:teuthology.orchestra.run.smithi122.stderr:+ NK=AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== 2024-09-11T08:09:44.164 INFO:teuthology.orchestra.run.smithi122.stderr:+ '[' AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== == AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== ']' 2024-09-11T08:09:44.164 INFO:teuthology.orchestra.run.smithi122.stderr:+ sleep 5 2024-09-11T08:09:44.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:09:44 smithi122 ceph-mon[28992]: pgmap v78: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:09:44.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:09:44 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/1679297274' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:09:44.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:09:44 smithi122 ceph-mon[39726]: pgmap v78: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:09:44.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:09:44 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/1679297274' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:09:44.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:09:44 smithi135 ceph-mon[32689]: pgmap v78: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:09:44.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:09:44 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/1679297274' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:09:46.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:09:46 smithi135 ceph-mon[32689]: pgmap v79: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:09:46.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:09:46 smithi122 ceph-mon[39726]: pgmap v79: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:09:46.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:09:46 smithi122 ceph-mon[28992]: pgmap v79: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:09:47.729 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:09:47 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:09:47.729 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:09:47 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:09:47.733 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:09:47 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:09:47.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:09:47 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:09:47.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:09:47 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:09:47.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:09:47 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:09:48.228 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:09:47 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: ::ffff:172.21.15.135 - - [11/Sep/2024:08:09:47] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:09:48.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:09:48 smithi135 ceph-mon[32689]: pgmap v80: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:09:48.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:09:48 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:09:48.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:09:48 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:09:48.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:09:48 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:09:48.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:09:48 smithi122 ceph-mon[28992]: pgmap v80: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:09:48.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:09:48 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:09:48.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:09:48 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:09:48.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:09:48 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:09:48.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:09:48 smithi122 ceph-mon[39726]: pgmap v80: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:09:48.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:09:48 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:09:48.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:09:48 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:09:48.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:09:48 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:09:49.166 INFO:teuthology.orchestra.run.smithi122.stderr:++ ceph auth get-key osd.0 2024-09-11T08:09:49.476 INFO:teuthology.orchestra.run.smithi122.stderr:+ NK=AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== 2024-09-11T08:09:49.476 INFO:teuthology.orchestra.run.smithi122.stderr:+ '[' AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== == AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== ']' 2024-09-11T08:09:49.476 INFO:teuthology.orchestra.run.smithi122.stderr:+ sleep 5 2024-09-11T08:09:50.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:09:50 smithi135 ceph-mon[32689]: pgmap v81: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:09:50.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:09:50 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/3862236974' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:09:50.733 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:09:50 smithi122 ceph-mon[28992]: pgmap v81: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:09:50.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:09:50 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/3862236974' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:09:50.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:09:50 smithi122 ceph-mon[39726]: pgmap v81: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:09:50.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:09:50 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/3862236974' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:09:52.424 INFO:journalctl@ceph.iscsi.iscsi.a.smithi135.stdout:Sep 11 08:09:52 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[59799]: debug there is no tcmu-runner data available 2024-09-11T08:09:52.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:09:52 smithi135 ceph-mon[32689]: pgmap v82: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:09:52.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:09:52 smithi122 ceph-mon[28992]: pgmap v82: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:09:52.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:09:52 smithi122 ceph-mon[39726]: pgmap v82: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:09:53.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:09:53 smithi135 ceph-mon[32689]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:09:53.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:09:53 smithi122 ceph-mon[28992]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:09:53.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:09:53 smithi122 ceph-mon[39726]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:09:53.734 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:09:53 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: ::ffff:172.21.15.135 - - [11/Sep/2024:08:09:53] "GET /metrics HTTP/1.1" 200 219512 "" "Prometheus/2.43.0" 2024-09-11T08:09:54.478 INFO:teuthology.orchestra.run.smithi122.stderr:++ ceph auth get-key osd.0 2024-09-11T08:09:54.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:09:54 smithi135 ceph-mon[32689]: pgmap v83: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:09:54.733 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:09:54 smithi122 ceph-mon[39726]: pgmap v83: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:09:54.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:09:54 smithi122 ceph-mon[28992]: pgmap v83: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:09:54.787 INFO:teuthology.orchestra.run.smithi122.stderr:+ NK=AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== 2024-09-11T08:09:54.787 INFO:teuthology.orchestra.run.smithi122.stderr:+ '[' AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== == AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== ']' 2024-09-11T08:09:54.787 INFO:teuthology.orchestra.run.smithi122.stderr:+ sleep 5 2024-09-11T08:09:55.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:09:55 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/3939759453' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:09:55.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:09:55 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/3939759453' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:09:55.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:09:55 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/3939759453' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:09:56.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:09:56 smithi135 ceph-mon[32689]: pgmap v84: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:09:56.733 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:09:56 smithi122 ceph-mon[28992]: pgmap v84: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:09:56.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:09:56 smithi122 ceph-mon[39726]: pgmap v84: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:09:58.228 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:09:57 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: ::ffff:172.21.15.135 - - [11/Sep/2024:08:09:57] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:09:58.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:09:58 smithi135 ceph-mon[32689]: pgmap v85: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:09:58.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:09:58 smithi122 ceph-mon[28992]: pgmap v85: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:09:58.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:09:58 smithi122 ceph-mon[39726]: pgmap v85: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:09:59.788 INFO:teuthology.orchestra.run.smithi122.stderr:++ ceph auth get-key osd.0 2024-09-11T08:10:00.099 INFO:teuthology.orchestra.run.smithi122.stderr:+ NK=AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== 2024-09-11T08:10:00.099 INFO:teuthology.orchestra.run.smithi122.stderr:+ '[' AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== == AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== ']' 2024-09-11T08:10:00.099 INFO:teuthology.orchestra.run.smithi122.stderr:+ sleep 5 2024-09-11T08:10:00.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:10:00 smithi135 ceph-mon[32689]: pgmap v86: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:10:00.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:10:00 smithi135 ceph-mon[32689]: overall HEALTH_OK 2024-09-11T08:10:00.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:10:00 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/3191013401' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:10:00.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:10:00 smithi122 ceph-mon[28992]: pgmap v86: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:10:00.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:10:00 smithi122 ceph-mon[28992]: overall HEALTH_OK 2024-09-11T08:10:00.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:10:00 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/3191013401' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:10:00.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:10:00 smithi122 ceph-mon[39726]: pgmap v86: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:10:00.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:10:00 smithi122 ceph-mon[39726]: overall HEALTH_OK 2024-09-11T08:10:00.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:10:00 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/3191013401' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:10:02.450 INFO:journalctl@ceph.iscsi.iscsi.a.smithi135.stdout:Sep 11 08:10:02 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[59799]: debug there is no tcmu-runner data available 2024-09-11T08:10:02.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:10:02 smithi135 ceph-mon[32689]: pgmap v87: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:10:02.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:10:02 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:10:02.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:10:02 smithi122 ceph-mon[28992]: pgmap v87: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:10:02.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:10:02 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:10:02.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:10:02 smithi122 ceph-mon[39726]: pgmap v87: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:10:02.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:10:02 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:10:03.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:10:03 smithi135 ceph-mon[32689]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:10:03.733 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:10:03 smithi122 ceph-mon[28992]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:10:03.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:10:03 smithi122 ceph-mon[39726]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:10:03.734 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:10:03 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: ::ffff:172.21.15.135 - - [11/Sep/2024:08:10:03] "GET /metrics HTTP/1.1" 200 219513 "" "Prometheus/2.43.0" 2024-09-11T08:10:04.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:10:04 smithi135 ceph-mon[32689]: pgmap v88: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:10:04.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:10:04 smithi122 ceph-mon[28992]: pgmap v88: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:10:04.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:10:04 smithi122 ceph-mon[39726]: pgmap v88: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:10:05.101 INFO:teuthology.orchestra.run.smithi122.stderr:++ ceph auth get-key osd.0 2024-09-11T08:10:05.412 INFO:teuthology.orchestra.run.smithi122.stderr:+ NK=AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== 2024-09-11T08:10:05.412 INFO:teuthology.orchestra.run.smithi122.stderr:+ '[' AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== == AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== ']' 2024-09-11T08:10:05.412 INFO:teuthology.orchestra.run.smithi122.stderr:+ sleep 5 2024-09-11T08:10:05.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:10:05 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/1911819451' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:10:05.733 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:10:05 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/1911819451' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:10:05.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:10:05 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/1911819451' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:10:06.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:10:06 smithi135 ceph-mon[32689]: pgmap v89: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:10:06.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:10:06 smithi122 ceph-mon[28992]: pgmap v89: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:10:06.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:10:06 smithi122 ceph-mon[39726]: pgmap v89: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:10:08.228 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:10:07 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: ::ffff:172.21.15.135 - - [11/Sep/2024:08:10:07] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:10:08.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:10:08 smithi135 ceph-mon[32689]: pgmap v90: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:10:08.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:10:08 smithi122 ceph-mon[28992]: pgmap v90: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:10:08.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:10:08 smithi122 ceph-mon[39726]: pgmap v90: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:10:10.414 INFO:teuthology.orchestra.run.smithi122.stderr:++ ceph auth get-key osd.0 2024-09-11T08:10:10.727 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:10:10 smithi135 ceph-mon[32689]: pgmap v91: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:10:10.731 INFO:teuthology.orchestra.run.smithi122.stderr:+ NK=AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== 2024-09-11T08:10:10.732 INFO:teuthology.orchestra.run.smithi122.stderr:+ '[' AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== == AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== ']' 2024-09-11T08:10:10.732 INFO:teuthology.orchestra.run.smithi122.stderr:+ sleep 5 2024-09-11T08:10:10.732 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:10:10 smithi122 ceph-mon[28992]: pgmap v91: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:10:10.732 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:10:10 smithi122 ceph-mon[39726]: pgmap v91: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:10:11.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:10:11 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/2492063849' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:10:11.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:10:11 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/2492063849' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:10:11.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:10:11 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/2492063849' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:10:12.476 INFO:journalctl@ceph.iscsi.iscsi.a.smithi135.stdout:Sep 11 08:10:12 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[59799]: debug there is no tcmu-runner data available 2024-09-11T08:10:12.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:10:12 smithi135 ceph-mon[32689]: pgmap v92: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:10:12.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:10:12 smithi122 ceph-mon[28992]: pgmap v92: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:10:12.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:10:12 smithi122 ceph-mon[39726]: pgmap v92: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:10:13.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:10:13 smithi122 ceph-mon[39726]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:10:13.735 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:10:13 smithi122 ceph-mon[28992]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:10:13.735 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:10:13 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: ::ffff:172.21.15.135 - - [11/Sep/2024:08:10:13] "GET /metrics HTTP/1.1" 200 219513 "" "Prometheus/2.43.0" 2024-09-11T08:10:13.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:10:13 smithi135 ceph-mon[32689]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:10:14.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:10:14 smithi122 ceph-mon[28992]: pgmap v93: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:10:14.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:10:14 smithi122 ceph-mon[39726]: pgmap v93: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:10:14.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:10:14 smithi135 ceph-mon[32689]: pgmap v93: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:10:15.734 INFO:teuthology.orchestra.run.smithi122.stderr:++ ceph auth get-key osd.0 2024-09-11T08:10:16.044 INFO:teuthology.orchestra.run.smithi122.stderr:+ NK=AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== 2024-09-11T08:10:16.045 INFO:teuthology.orchestra.run.smithi122.stderr:+ '[' AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== == AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== ']' 2024-09-11T08:10:16.045 INFO:teuthology.orchestra.run.smithi122.stderr:+ sleep 5 2024-09-11T08:10:16.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:10:16 smithi135 ceph-mon[32689]: pgmap v94: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:10:16.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:10:16 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/2601892469' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:10:16.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:10:16 smithi122 ceph-mon[39726]: pgmap v94: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:10:16.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:10:16 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/2601892469' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:10:16.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:10:16 smithi122 ceph-mon[28992]: pgmap v94: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:10:16.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:10:16 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/2601892469' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:10:17.935 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:10:17 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:10:17.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:10:17 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:10:17.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:10:17 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:10:18.228 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:10:17 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: ::ffff:172.21.15.135 - - [11/Sep/2024:08:10:17] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:10:18.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:10:18 smithi135 ceph-mon[32689]: pgmap v95: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:10:18.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:10:18 smithi122 ceph-mon[28992]: pgmap v95: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:10:18.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:10:18 smithi122 ceph-mon[39726]: pgmap v95: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:10:20.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:10:20 smithi135 ceph-mon[32689]: pgmap v96: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:10:20.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:10:20 smithi122 ceph-mon[39726]: pgmap v96: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:10:20.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:10:20 smithi122 ceph-mon[28992]: pgmap v96: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:10:21.047 INFO:teuthology.orchestra.run.smithi122.stderr:++ ceph auth get-key osd.0 2024-09-11T08:10:21.357 INFO:teuthology.orchestra.run.smithi122.stderr:+ NK=AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== 2024-09-11T08:10:21.358 INFO:teuthology.orchestra.run.smithi122.stderr:+ '[' AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== == AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== ']' 2024-09-11T08:10:21.358 INFO:teuthology.orchestra.run.smithi122.stderr:+ sleep 5 2024-09-11T08:10:21.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:10:21 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/522527882' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:10:21.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:10:21 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/522527882' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:10:21.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:10:21 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/522527882' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:10:22.478 INFO:journalctl@ceph.iscsi.iscsi.a.smithi135.stdout:Sep 11 08:10:22 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[59799]: debug there is no tcmu-runner data available 2024-09-11T08:10:22.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:10:22 smithi135 ceph-mon[32689]: pgmap v97: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:10:22.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:10:22 smithi122 ceph-mon[28992]: pgmap v97: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:10:22.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:10:22 smithi122 ceph-mon[39726]: pgmap v97: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:10:23.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:10:23 smithi122 ceph-mon[28992]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:10:23.734 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:10:23 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: ::ffff:172.21.15.135 - - [11/Sep/2024:08:10:23] "GET /metrics HTTP/1.1" 200 219509 "" "Prometheus/2.43.0" 2024-09-11T08:10:23.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:10:23 smithi122 ceph-mon[39726]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:10:23.980 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:10:23 smithi135 ceph-mon[32689]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:10:24.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:10:24 smithi135 ceph-mon[32689]: pgmap v98: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:10:24.995 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:10:24 smithi122 ceph-mon[28992]: pgmap v98: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:10:24.995 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:10:24 smithi122 ceph-mon[39726]: pgmap v98: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:10:26.359 INFO:teuthology.orchestra.run.smithi122.stderr:++ ceph auth get-key osd.0 2024-09-11T08:10:26.681 INFO:teuthology.orchestra.run.smithi122.stderr:+ NK=AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== 2024-09-11T08:10:26.681 INFO:teuthology.orchestra.run.smithi122.stderr:+ '[' AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== == AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== ']' 2024-09-11T08:10:26.681 INFO:teuthology.orchestra.run.smithi122.stderr:+ sleep 5 2024-09-11T08:10:26.682 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:10:26 smithi122 ceph-mon[28992]: pgmap v99: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:10:26.682 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:10:26 smithi122 ceph-mon[39726]: pgmap v99: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:10:26.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:10:26 smithi135 ceph-mon[32689]: pgmap v99: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:10:27.935 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:10:27 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/2434538392' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:10:27.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:10:27 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/2434538392' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:10:27.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:10:27 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/2434538392' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:10:28.228 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:10:27 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: ::ffff:172.21.15.135 - - [11/Sep/2024:08:10:27] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:10:28.885 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:10:28 smithi122 ceph-mon[39726]: pgmap v100: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:10:28.886 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:10:28 smithi122 ceph-mon[28992]: pgmap v100: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:10:28.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:10:28 smithi135 ceph-mon[32689]: pgmap v100: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:10:30.780 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:10:30 smithi135 ceph-mon[32689]: pgmap v101: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:10:30.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:10:30 smithi122 ceph-mon[39726]: pgmap v101: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:10:30.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:10:30 smithi122 ceph-mon[28992]: pgmap v101: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:10:31.684 INFO:teuthology.orchestra.run.smithi122.stderr:++ ceph auth get-key osd.0 2024-09-11T08:10:31.995 INFO:teuthology.orchestra.run.smithi122.stderr:+ NK=AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== 2024-09-11T08:10:31.995 INFO:teuthology.orchestra.run.smithi122.stderr:+ '[' AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== == AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== ']' 2024-09-11T08:10:31.996 INFO:teuthology.orchestra.run.smithi122.stderr:+ sleep 5 2024-09-11T08:10:32.478 INFO:journalctl@ceph.iscsi.iscsi.a.smithi135.stdout:Sep 11 08:10:32 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[59799]: debug there is no tcmu-runner data available 2024-09-11T08:10:32.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:10:32 smithi135 ceph-mon[32689]: pgmap v102: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:10:32.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:10:32 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/2783387452' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:10:32.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:10:32 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:10:32.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:10:32 smithi122 ceph-mon[28992]: pgmap v102: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:10:32.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:10:32 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/2783387452' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:10:32.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:10:32 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:10:32.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:10:32 smithi122 ceph-mon[39726]: pgmap v102: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:10:32.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:10:32 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/2783387452' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:10:32.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:10:32 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:10:33.535 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:10:33 smithi122 ceph-mon[28992]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:10:33.535 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:10:33 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: ::ffff:172.21.15.135 - - [11/Sep/2024:08:10:33] "GET /metrics HTTP/1.1" 200 219483 "" "Prometheus/2.43.0" 2024-09-11T08:10:33.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:10:33 smithi135 ceph-mon[32689]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:10:33.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:10:33 smithi122 ceph-mon[39726]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:10:34.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:10:34 smithi135 ceph-mon[32689]: pgmap v103: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:10:34.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:10:34 smithi122 ceph-mon[28992]: pgmap v103: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:10:34.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:10:34 smithi122 ceph-mon[39726]: pgmap v103: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:10:36.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:10:36 smithi135 ceph-mon[32689]: pgmap v104: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:10:36.983 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:10:36 smithi122 ceph-mon[28992]: pgmap v104: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:10:36.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:10:36 smithi122 ceph-mon[39726]: pgmap v104: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:10:36.997 INFO:teuthology.orchestra.run.smithi122.stderr:++ ceph auth get-key osd.0 2024-09-11T08:10:37.314 INFO:teuthology.orchestra.run.smithi122.stderr:+ NK=AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== 2024-09-11T08:10:37.315 INFO:teuthology.orchestra.run.smithi122.stderr:+ '[' AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== == AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== ']' 2024-09-11T08:10:37.315 INFO:teuthology.orchestra.run.smithi122.stderr:+ sleep 5 2024-09-11T08:10:37.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:10:37 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/6799432' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:10:37.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:10:37 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/6799432' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:10:37.936 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:10:37 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/6799432' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:10:38.228 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:10:37 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: ::ffff:172.21.15.135 - - [11/Sep/2024:08:10:37] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:10:38.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:10:38 smithi135 ceph-mon[32689]: pgmap v105: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:10:38.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:10:38 smithi122 ceph-mon[28992]: pgmap v105: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:10:38.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:10:38 smithi122 ceph-mon[39726]: pgmap v105: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:10:40.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:10:40 smithi135 ceph-mon[32689]: pgmap v106: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:10:40.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:10:40 smithi122 ceph-mon[39726]: pgmap v106: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:10:40.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:10:40 smithi122 ceph-mon[28992]: pgmap v106: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:10:42.317 INFO:teuthology.orchestra.run.smithi122.stderr:++ ceph auth get-key osd.0 2024-09-11T08:10:42.478 INFO:journalctl@ceph.iscsi.iscsi.a.smithi135.stdout:Sep 11 08:10:42 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[59799]: debug there is no tcmu-runner data available 2024-09-11T08:10:42.628 INFO:teuthology.orchestra.run.smithi122.stderr:+ NK=AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== 2024-09-11T08:10:42.628 INFO:teuthology.orchestra.run.smithi122.stderr:+ '[' AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== == AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== ']' 2024-09-11T08:10:42.628 INFO:teuthology.orchestra.run.smithi122.stderr:+ sleep 5 2024-09-11T08:10:42.629 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:10:42 smithi122 ceph-mon[28992]: pgmap v107: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:10:42.629 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:10:42 smithi122 ceph-mon[39726]: pgmap v107: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:10:42.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:10:42 smithi135 ceph-mon[32689]: pgmap v107: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:10:43.559 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:10:43 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: ::ffff:172.21.15.135 - - [11/Sep/2024:08:10:43] "GET /metrics HTTP/1.1" 200 219483 "" "Prometheus/2.43.0" 2024-09-11T08:10:43.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:10:43 smithi135 ceph-mon[32689]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:10:43.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:10:43 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/2984093431' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:10:43.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:10:43 smithi122 ceph-mon[39726]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:10:43.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:10:43 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/2984093431' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:10:43.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:10:43 smithi122 ceph-mon[28992]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:10:43.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:10:43 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/2984093431' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:10:44.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:10:44 smithi135 ceph-mon[32689]: pgmap v108: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:10:44.983 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:10:44 smithi122 ceph-mon[28992]: pgmap v108: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:10:44.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:10:44 smithi122 ceph-mon[39726]: pgmap v108: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:10:46.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:10:46 smithi135 ceph-mon[32689]: pgmap v109: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:10:46.983 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:10:46 smithi122 ceph-mon[39726]: pgmap v109: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:10:46.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:10:46 smithi122 ceph-mon[28992]: pgmap v109: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:10:47.630 INFO:teuthology.orchestra.run.smithi122.stderr:++ ceph auth get-key osd.0 2024-09-11T08:10:47.935 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:10:47 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:10:47.952 INFO:teuthology.orchestra.run.smithi122.stderr:+ NK=AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== 2024-09-11T08:10:47.953 INFO:teuthology.orchestra.run.smithi122.stderr:+ '[' AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== == AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== ']' 2024-09-11T08:10:47.953 INFO:teuthology.orchestra.run.smithi122.stderr:+ sleep 5 2024-09-11T08:10:47.953 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:10:47 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:10:47.953 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:10:47 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:10:48.228 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:10:47 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: ::ffff:172.21.15.135 - - [11/Sep/2024:08:10:47] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:10:48.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:10:48 smithi135 ceph-mon[32689]: pgmap v110: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:10:48.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:10:48 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:10:48.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:10:48 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/2138783598' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:10:48.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:10:48 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:10:48.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:10:48 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:10:48.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:10:48 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:10:48.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:10:48 smithi122 ceph-mon[28992]: pgmap v110: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:10:48.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:10:48 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:10:48.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:10:48 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/2138783598' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:10:48.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:10:48 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:10:48.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:10:48 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:10:48.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:10:48 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:10:48.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:10:48 smithi122 ceph-mon[39726]: pgmap v110: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:10:48.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:10:48 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:10:48.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:10:48 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/2138783598' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:10:48.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:10:48 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:10:48.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:10:48 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:10:48.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:10:48 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:10:50.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:10:50 smithi135 ceph-mon[32689]: pgmap v111: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:10:50.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:10:50 smithi122 ceph-mon[28992]: pgmap v111: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:10:50.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:10:50 smithi122 ceph-mon[39726]: pgmap v111: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:10:52.478 INFO:journalctl@ceph.iscsi.iscsi.a.smithi135.stdout:Sep 11 08:10:52 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[59799]: debug there is no tcmu-runner data available 2024-09-11T08:10:52.955 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:10:52 smithi122 ceph-mon[39726]: pgmap v112: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-11T08:10:52.955 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:10:52 smithi122 ceph-mon[28992]: pgmap v112: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-11T08:10:52.955 INFO:teuthology.orchestra.run.smithi122.stderr:++ ceph auth get-key osd.0 2024-09-11T08:10:52.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:10:52 smithi135 ceph-mon[32689]: pgmap v112: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-11T08:10:53.264 INFO:teuthology.orchestra.run.smithi122.stderr:+ NK=AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== 2024-09-11T08:10:53.264 INFO:teuthology.orchestra.run.smithi122.stderr:+ '[' AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== == AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== ']' 2024-09-11T08:10:53.265 INFO:teuthology.orchestra.run.smithi122.stderr:+ sleep 5 2024-09-11T08:10:53.592 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:10:53 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: ::ffff:172.21.15.135 - - [11/Sep/2024:08:10:53] "GET /metrics HTTP/1.1" 200 219481 "" "Prometheus/2.43.0" 2024-09-11T08:10:53.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:10:53 smithi135 ceph-mon[32689]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:10:53.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:10:53 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/2150741981' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:10:53.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:10:53 smithi122 ceph-mon[28992]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:10:53.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:10:53 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/2150741981' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:10:53.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:10:53 smithi122 ceph-mon[39726]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:10:53.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:10:53 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/2150741981' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:10:54.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:10:54 smithi135 ceph-mon[32689]: pgmap v113: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:10:54.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:10:54 smithi122 ceph-mon[28992]: pgmap v113: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:10:54.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:10:54 smithi122 ceph-mon[39726]: pgmap v113: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:10:56.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:10:56 smithi135 ceph-mon[32689]: pgmap v114: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-11T08:10:56.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:10:56 smithi122 ceph-mon[28992]: pgmap v114: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-11T08:10:56.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:10:56 smithi122 ceph-mon[39726]: pgmap v114: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-11T08:10:58.228 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:10:57 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: ::ffff:172.21.15.135 - - [11/Sep/2024:08:10:57] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:10:58.266 INFO:teuthology.orchestra.run.smithi122.stderr:++ ceph auth get-key osd.0 2024-09-11T08:10:58.578 INFO:teuthology.orchestra.run.smithi122.stderr:+ NK=AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== 2024-09-11T08:10:58.578 INFO:teuthology.orchestra.run.smithi122.stderr:+ '[' AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== == AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== ']' 2024-09-11T08:10:58.578 INFO:teuthology.orchestra.run.smithi122.stderr:+ sleep 5 2024-09-11T08:10:58.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:10:58 smithi135 ceph-mon[32689]: pgmap v115: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-11T08:10:58.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:10:58 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/141843602' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:10:58.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:10:58 smithi122 ceph-mon[28992]: pgmap v115: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-11T08:10:58.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:10:58 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/141843602' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:10:58.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:10:58 smithi122 ceph-mon[39726]: pgmap v115: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-11T08:10:58.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:10:58 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/141843602' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:11:00.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:11:00 smithi135 ceph-mon[32689]: pgmap v116: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:11:00.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:11:00 smithi122 ceph-mon[39726]: pgmap v116: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:11:00.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:11:00 smithi122 ceph-mon[28992]: pgmap v116: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:11:02.478 INFO:journalctl@ceph.iscsi.iscsi.a.smithi135.stdout:Sep 11 08:11:02 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[59799]: debug there is no tcmu-runner data available 2024-09-11T08:11:02.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:11:02 smithi135 ceph-mon[32689]: pgmap v117: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:11:02.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:11:02 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:11:02.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:11:02 smithi122 ceph-mon[28992]: pgmap v117: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:11:02.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:11:02 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:11:02.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:11:02 smithi122 ceph-mon[39726]: pgmap v117: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:11:02.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:11:02 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:11:03.580 INFO:teuthology.orchestra.run.smithi122.stderr:++ ceph auth get-key osd.0 2024-09-11T08:11:03.581 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:11:03 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: ::ffff:172.21.15.135 - - [11/Sep/2024:08:11:03] "GET /metrics HTTP/1.1" 200 219493 "" "Prometheus/2.43.0" 2024-09-11T08:11:03.895 INFO:teuthology.orchestra.run.smithi122.stderr:+ NK=AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== 2024-09-11T08:11:03.895 INFO:teuthology.orchestra.run.smithi122.stderr:+ '[' AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== == AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== ']' 2024-09-11T08:11:03.895 INFO:teuthology.orchestra.run.smithi122.stderr:+ sleep 5 2024-09-11T08:11:03.896 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:11:03 smithi122 ceph-mon[28992]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:11:03.896 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:11:03 smithi122 ceph-mon[39726]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:11:03.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:11:03 smithi135 ceph-mon[32689]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:11:04.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:11:04 smithi135 ceph-mon[32689]: pgmap v118: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:11:04.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:11:04 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/2208166924' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:11:04.983 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:11:04 smithi122 ceph-mon[39726]: pgmap v118: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:11:04.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:11:04 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/2208166924' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:11:04.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:11:04 smithi122 ceph-mon[28992]: pgmap v118: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:11:04.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:11:04 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/2208166924' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:11:06.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:11:06 smithi135 ceph-mon[32689]: pgmap v119: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:11:06.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:11:06 smithi122 ceph-mon[39726]: pgmap v119: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:11:06.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:11:06 smithi122 ceph-mon[28992]: pgmap v119: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:11:08.228 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:11:07 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: ::ffff:172.21.15.135 - - [11/Sep/2024:08:11:07] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:11:08.897 INFO:teuthology.orchestra.run.smithi122.stderr:++ ceph auth get-key osd.0 2024-09-11T08:11:08.898 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:11:08 smithi122 ceph-mon[28992]: pgmap v120: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:11:08.898 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:11:08 smithi122 ceph-mon[39726]: pgmap v120: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:11:08.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:11:08 smithi135 ceph-mon[32689]: pgmap v120: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:11:09.208 INFO:teuthology.orchestra.run.smithi122.stderr:+ NK=AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== 2024-09-11T08:11:09.209 INFO:teuthology.orchestra.run.smithi122.stderr:+ '[' AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== == AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== ']' 2024-09-11T08:11:09.209 INFO:teuthology.orchestra.run.smithi122.stderr:+ sleep 5 2024-09-11T08:11:09.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:11:09 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/3132314814' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:11:09.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:11:09 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/3132314814' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:11:09.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:11:09 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/3132314814' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:11:10.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:11:10 smithi135 ceph-mon[32689]: pgmap v121: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:11:10.983 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:11:10 smithi122 ceph-mon[28992]: pgmap v121: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:11:10.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:11:10 smithi122 ceph-mon[39726]: pgmap v121: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:11:11.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:11:11 smithi135 ceph-mon[32689]: pgmap v122: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:11:11.983 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:11:11 smithi122 ceph-mon[28992]: pgmap v122: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:11:11.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:11:11 smithi122 ceph-mon[39726]: pgmap v122: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:11:12.478 INFO:journalctl@ceph.iscsi.iscsi.a.smithi135.stdout:Sep 11 08:11:12 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[59799]: debug there is no tcmu-runner data available 2024-09-11T08:11:12.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:11:12 smithi135 ceph-mon[32689]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:11:12.983 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:11:12 smithi122 ceph-mon[28992]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:11:12.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:11:12 smithi122 ceph-mon[39726]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:11:13.693 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:11:13 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: ::ffff:172.21.15.135 - - [11/Sep/2024:08:11:13] "GET /metrics HTTP/1.1" 200 219493 "" "Prometheus/2.43.0" 2024-09-11T08:11:13.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:11:13 smithi135 ceph-mon[32689]: pgmap v123: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:11:13.983 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:11:13 smithi122 ceph-mon[28992]: pgmap v123: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:11:13.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:11:13 smithi122 ceph-mon[39726]: pgmap v123: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:11:14.210 INFO:teuthology.orchestra.run.smithi122.stderr:++ ceph auth get-key osd.0 2024-09-11T08:11:14.530 INFO:teuthology.orchestra.run.smithi122.stderr:+ NK=AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== 2024-09-11T08:11:14.530 INFO:teuthology.orchestra.run.smithi122.stderr:+ '[' AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== == AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== ']' 2024-09-11T08:11:14.530 INFO:teuthology.orchestra.run.smithi122.stderr:+ sleep 5 2024-09-11T08:11:14.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:11:14 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/3540560858' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:11:14.983 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:11:14 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/3540560858' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:11:14.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:11:14 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/3540560858' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:11:15.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:11:15 smithi135 ceph-mon[32689]: pgmap v124: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:11:15.983 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:11:15 smithi122 ceph-mon[28992]: pgmap v124: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:11:15.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:11:15 smithi122 ceph-mon[39726]: pgmap v124: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:11:17.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:11:17 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:11:17.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:11:17 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:11:17.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:11:17 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:11:18.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:11:18 smithi135 ceph-mon[32689]: pgmap v125: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:11:18.228 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:11:17 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: ::ffff:172.21.15.135 - - [11/Sep/2024:08:11:17] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:11:18.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:11:18 smithi122 ceph-mon[28992]: pgmap v125: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:11:18.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:11:18 smithi122 ceph-mon[39726]: pgmap v125: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:11:19.531 INFO:teuthology.orchestra.run.smithi122.stderr:++ ceph auth get-key osd.0 2024-09-11T08:11:19.845 INFO:teuthology.orchestra.run.smithi122.stderr:+ NK=AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== 2024-09-11T08:11:19.845 INFO:teuthology.orchestra.run.smithi122.stderr:+ '[' AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== == AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== ']' 2024-09-11T08:11:19.846 INFO:teuthology.orchestra.run.smithi122.stderr:+ sleep 5 2024-09-11T08:11:20.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:11:20 smithi135 ceph-mon[32689]: pgmap v126: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:11:20.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:11:20 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/3722436940' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:11:20.483 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:11:20 smithi122 ceph-mon[28992]: pgmap v126: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:11:20.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:11:20 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/3722436940' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:11:20.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:11:20 smithi122 ceph-mon[39726]: pgmap v126: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:11:20.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:11:20 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/3722436940' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:11:22.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:11:22 smithi135 ceph-mon[32689]: pgmap v127: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-11T08:11:22.478 INFO:journalctl@ceph.iscsi.iscsi.a.smithi135.stdout:Sep 11 08:11:22 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[59799]: debug there is no tcmu-runner data available 2024-09-11T08:11:22.483 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:11:22 smithi122 ceph-mon[28992]: pgmap v127: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-11T08:11:22.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:11:22 smithi122 ceph-mon[39726]: pgmap v127: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-11T08:11:23.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:11:23 smithi135 ceph-mon[32689]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:11:23.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:11:23 smithi122 ceph-mon[28992]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:11:23.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:11:23 smithi122 ceph-mon[39726]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:11:23.484 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:11:23 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: ::ffff:172.21.15.135 - - [11/Sep/2024:08:11:23] "GET /metrics HTTP/1.1" 200 219492 "" "Prometheus/2.43.0" 2024-09-11T08:11:24.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:11:24 smithi135 ceph-mon[32689]: pgmap v128: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:11:24.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:11:24 smithi122 ceph-mon[28992]: pgmap v128: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:11:24.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:11:24 smithi122 ceph-mon[39726]: pgmap v128: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:11:24.847 INFO:teuthology.orchestra.run.smithi122.stderr:++ ceph auth get-key osd.0 2024-09-11T08:11:25.161 INFO:teuthology.orchestra.run.smithi122.stderr:+ NK=AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== 2024-09-11T08:11:25.161 INFO:teuthology.orchestra.run.smithi122.stderr:+ '[' AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== == AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== ']' 2024-09-11T08:11:25.161 INFO:teuthology.orchestra.run.smithi122.stderr:+ sleep 5 2024-09-11T08:11:25.411 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:11:25 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/3119294638' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:11:25.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:11:25 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/3119294638' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:11:25.733 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:11:25 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/3119294638' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:11:26.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:11:26 smithi135 ceph-mon[32689]: pgmap v129: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-11T08:11:26.733 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:11:26 smithi122 ceph-mon[39726]: pgmap v129: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-11T08:11:26.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:11:26 smithi122 ceph-mon[28992]: pgmap v129: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-11T08:11:28.228 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:11:27 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: ::ffff:172.21.15.135 - - [11/Sep/2024:08:11:27] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:11:28.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:11:28 smithi135 ceph-mon[32689]: pgmap v130: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-11T08:11:28.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:11:28 smithi122 ceph-mon[28992]: pgmap v130: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-11T08:11:28.735 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:11:28 smithi122 ceph-mon[39726]: pgmap v130: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-11T08:11:30.163 INFO:teuthology.orchestra.run.smithi122.stderr:++ ceph auth get-key osd.0 2024-09-11T08:11:30.487 INFO:teuthology.orchestra.run.smithi122.stderr:+ NK=AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== 2024-09-11T08:11:30.487 INFO:teuthology.orchestra.run.smithi122.stderr:+ '[' AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== == AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== ']' 2024-09-11T08:11:30.487 INFO:teuthology.orchestra.run.smithi122.stderr:+ sleep 5 2024-09-11T08:11:30.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:11:30 smithi135 ceph-mon[32689]: pgmap v131: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:11:30.733 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:11:30 smithi122 ceph-mon[28992]: pgmap v131: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:11:30.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:11:30 smithi122 ceph-mon[39726]: pgmap v131: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:11:31.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:11:31 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/229825552' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:11:31.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:11:31 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/229825552' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:11:31.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:11:31 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/229825552' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:11:32.423 INFO:journalctl@ceph.iscsi.iscsi.a.smithi135.stdout:Sep 11 08:11:32 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[59799]: debug there is no tcmu-runner data available 2024-09-11T08:11:32.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:11:32 smithi135 ceph-mon[32689]: pgmap v132: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:11:32.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:11:32 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:11:32.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:11:32 smithi122 ceph-mon[39726]: pgmap v132: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:11:32.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:11:32 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:11:32.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:11:32 smithi122 ceph-mon[28992]: pgmap v132: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:11:32.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:11:32 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:11:33.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:11:33 smithi135 ceph-mon[32689]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:11:33.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:11:33 smithi122 ceph-mon[28992]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:11:33.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:11:33 smithi122 ceph-mon[39726]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:11:33.734 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:11:33 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: ::ffff:172.21.15.135 - - [11/Sep/2024:08:11:33] "GET /metrics HTTP/1.1" 200 219495 "" "Prometheus/2.43.0" 2024-09-11T08:11:34.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:11:34 smithi135 ceph-mon[32689]: pgmap v133: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:11:34.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:11:34 smithi122 ceph-mon[28992]: pgmap v133: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:11:34.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:11:34 smithi122 ceph-mon[39726]: pgmap v133: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:11:35.488 INFO:teuthology.orchestra.run.smithi122.stderr:++ ceph auth get-key osd.0 2024-09-11T08:11:35.800 INFO:teuthology.orchestra.run.smithi122.stderr:+ NK=AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== 2024-09-11T08:11:35.801 INFO:teuthology.orchestra.run.smithi122.stderr:+ '[' AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== == AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== ']' 2024-09-11T08:11:35.801 INFO:teuthology.orchestra.run.smithi122.stderr:+ sleep 5 2024-09-11T08:11:36.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:11:36 smithi135 ceph-mon[32689]: pgmap v134: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:11:36.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:11:36 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/1666248244' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:11:36.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:11:36 smithi122 ceph-mon[28992]: pgmap v134: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:11:36.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:11:36 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/1666248244' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:11:36.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:11:36 smithi122 ceph-mon[39726]: pgmap v134: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:11:36.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:11:36 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/1666248244' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:11:38.228 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:11:37 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: ::ffff:172.21.15.135 - - [11/Sep/2024:08:11:37] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:11:38.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:11:38 smithi135 ceph-mon[32689]: pgmap v135: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:11:38.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:11:38 smithi122 ceph-mon[39726]: pgmap v135: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:11:38.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:11:38 smithi122 ceph-mon[28992]: pgmap v135: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:11:40.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:11:40 smithi135 ceph-mon[32689]: pgmap v136: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:11:40.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:11:40 smithi122 ceph-mon[39726]: pgmap v136: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:11:40.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:11:40 smithi122 ceph-mon[28992]: pgmap v136: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:11:40.803 INFO:teuthology.orchestra.run.smithi122.stderr:++ ceph auth get-key osd.0 2024-09-11T08:11:41.113 INFO:teuthology.orchestra.run.smithi122.stderr:+ NK=AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== 2024-09-11T08:11:41.113 INFO:teuthology.orchestra.run.smithi122.stderr:+ '[' AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== == AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== ']' 2024-09-11T08:11:41.113 INFO:teuthology.orchestra.run.smithi122.stderr:+ sleep 5 2024-09-11T08:11:41.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:11:41 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/3638722605' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:11:41.733 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:11:41 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/3638722605' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:11:41.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:11:41 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/3638722605' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:11:42.446 INFO:journalctl@ceph.iscsi.iscsi.a.smithi135.stdout:Sep 11 08:11:42 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[59799]: debug there is no tcmu-runner data available 2024-09-11T08:11:42.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:11:42 smithi135 ceph-mon[32689]: pgmap v137: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:11:42.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:11:42 smithi122 ceph-mon[28992]: pgmap v137: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:11:42.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:11:42 smithi122 ceph-mon[39726]: pgmap v137: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:11:43.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:11:43 smithi135 ceph-mon[32689]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:11:43.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:11:43 smithi122 ceph-mon[28992]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:11:43.734 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:11:43 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: ::ffff:172.21.15.135 - - [11/Sep/2024:08:11:43] "GET /metrics HTTP/1.1" 200 219495 "" "Prometheus/2.43.0" 2024-09-11T08:11:43.735 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:11:43 smithi122 ceph-mon[39726]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:11:44.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:11:44 smithi135 ceph-mon[32689]: pgmap v138: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:11:44.733 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:11:44 smithi122 ceph-mon[28992]: pgmap v138: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:11:44.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:11:44 smithi122 ceph-mon[39726]: pgmap v138: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:11:46.115 INFO:teuthology.orchestra.run.smithi122.stderr:++ ceph auth get-key osd.0 2024-09-11T08:11:46.428 INFO:teuthology.orchestra.run.smithi122.stderr:+ NK=AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== 2024-09-11T08:11:46.429 INFO:teuthology.orchestra.run.smithi122.stderr:+ '[' AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== == AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== ']' 2024-09-11T08:11:46.429 INFO:teuthology.orchestra.run.smithi122.stderr:+ sleep 5 2024-09-11T08:11:46.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:11:46 smithi135 ceph-mon[32689]: pgmap v139: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:11:46.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:11:46 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/255080445' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:11:46.733 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:11:46 smithi122 ceph-mon[28992]: pgmap v139: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:11:46.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:11:46 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/255080445' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:11:46.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:11:46 smithi122 ceph-mon[39726]: pgmap v139: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:11:46.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:11:46 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/255080445' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:11:47.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:11:47 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:11:47.733 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:11:47 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:11:47.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:11:47 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:11:48.228 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:11:47 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: ::ffff:172.21.15.135 - - [11/Sep/2024:08:11:47] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:11:48.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:11:48 smithi135 ceph-mon[32689]: pgmap v140: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:11:48.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:11:48 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:11:48.733 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:11:48 smithi122 ceph-mon[28992]: pgmap v140: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:11:48.733 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:11:48 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:11:48.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:11:48 smithi122 ceph-mon[39726]: pgmap v140: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:11:48.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:11:48 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:11:49.729 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:11:49 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:11:49.729 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:11:49 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:11:49.729 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:11:49 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:11:49.740 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:11:49 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:11:49.740 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:11:49 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:11:49.740 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:11:49 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:11:49.740 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:11:49 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:11:49.740 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:11:49 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:11:49.740 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:11:49 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:11:50.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:11:50 smithi135 ceph-mon[32689]: pgmap v141: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:11:50.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:11:50 smithi122 ceph-mon[28992]: pgmap v141: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:11:50.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:11:50 smithi122 ceph-mon[39726]: pgmap v141: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:11:51.430 INFO:teuthology.orchestra.run.smithi122.stderr:++ ceph auth get-key osd.0 2024-09-11T08:11:51.745 INFO:teuthology.orchestra.run.smithi122.stderr:+ NK=AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== 2024-09-11T08:11:51.745 INFO:teuthology.orchestra.run.smithi122.stderr:+ '[' AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== == AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== ']' 2024-09-11T08:11:51.745 INFO:teuthology.orchestra.run.smithi122.stderr:+ sleep 5 2024-09-11T08:11:52.478 INFO:journalctl@ceph.iscsi.iscsi.a.smithi135.stdout:Sep 11 08:11:52 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[59799]: debug there is no tcmu-runner data available 2024-09-11T08:11:52.977 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:11:52 smithi135 ceph-mon[32689]: pgmap v142: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:11:52.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:11:52 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/3316224645' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:11:52.983 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:11:52 smithi122 ceph-mon[28992]: pgmap v142: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:11:52.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:11:52 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/3316224645' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:11:52.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:11:52 smithi122 ceph-mon[39726]: pgmap v142: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:11:52.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:11:52 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/3316224645' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:11:53.616 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:11:53 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: ::ffff:172.21.15.135 - - [11/Sep/2024:08:11:53] "GET /metrics HTTP/1.1" 200 219488 "" "Prometheus/2.43.0" 2024-09-11T08:11:53.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:11:53 smithi135 ceph-mon[32689]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:11:53.983 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:11:53 smithi122 ceph-mon[28992]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:11:53.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:11:53 smithi122 ceph-mon[39726]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:11:54.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:11:54 smithi135 ceph-mon[32689]: pgmap v143: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:11:54.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:11:54 smithi122 ceph-mon[28992]: pgmap v143: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:11:54.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:11:54 smithi122 ceph-mon[39726]: pgmap v143: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:11:56.746 INFO:teuthology.orchestra.run.smithi122.stderr:++ ceph auth get-key osd.0 2024-09-11T08:11:56.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:11:56 smithi135 ceph-mon[32689]: pgmap v144: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:11:56.983 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:11:56 smithi122 ceph-mon[39726]: pgmap v144: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:11:56.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:11:56 smithi122 ceph-mon[28992]: pgmap v144: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:11:57.058 INFO:teuthology.orchestra.run.smithi122.stdout:rotating key for osd.1 2024-09-11T08:11:57.059 INFO:teuthology.orchestra.run.smithi122.stderr:+ NK=AQD4T+FmuTk8DxAAqyL82zo4OUIU4gja2VjuOw== 2024-09-11T08:11:57.059 INFO:teuthology.orchestra.run.smithi122.stderr:+ '[' AQDMTuFmbaOYNBAAmAega7eBefX17iIluysy8Q== == AQD4T+FmuTk8DxAAqyL82zo4OUIU4gja2VjuOw== ']' 2024-09-11T08:11:57.059 INFO:teuthology.orchestra.run.smithi122.stderr:+ for f in osd.0 osd.1 osd.2 osd.3 osd.4 osd.5 osd.6 osd.7 mgr.y mgr.x 2024-09-11T08:11:57.059 INFO:teuthology.orchestra.run.smithi122.stderr:+ echo 'rotating key for osd.1' 2024-09-11T08:11:57.059 INFO:teuthology.orchestra.run.smithi122.stderr:++ ceph auth get-key osd.1 2024-09-11T08:11:57.360 INFO:teuthology.orchestra.run.smithi122.stderr:+ K=AQDkTuFmacylFRAA4RuBLRnWI1X/+CLMkDzjMQ== 2024-09-11T08:11:57.360 INFO:teuthology.orchestra.run.smithi122.stderr:+ NK=AQDkTuFmacylFRAA4RuBLRnWI1X/+CLMkDzjMQ== 2024-09-11T08:11:57.360 INFO:teuthology.orchestra.run.smithi122.stderr:+ ceph orch daemon rotate-key osd.1 2024-09-11T08:11:57.617 INFO:teuthology.orchestra.run.smithi122.stdout:Scheduled to rotate-key osd.1 on host 'smithi122' 2024-09-11T08:11:57.629 INFO:teuthology.orchestra.run.smithi122.stderr:+ '[' AQDkTuFmacylFRAA4RuBLRnWI1X/+CLMkDzjMQ== == AQDkTuFmacylFRAA4RuBLRnWI1X/+CLMkDzjMQ== ']' 2024-09-11T08:11:57.629 INFO:teuthology.orchestra.run.smithi122.stderr:+ sleep 5 2024-09-11T08:11:57.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:11:57 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/4028291139' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:11:57.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:11:57 smithi122 ceph-mon[39726]: pgmap v145: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:11:57.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:11:57 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/2226302634' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-11T08:11:57.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:11:57 smithi122 ceph-mon[39726]: from='client.14979 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:11:57.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:11:57 smithi122 ceph-mon[39726]: Schedule rotate-key daemon osd.1 2024-09-11T08:11:57.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:11:57 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:11:57.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:11:57 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:11:57.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:11:57 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:11:57.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:11:57 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:11:57.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:11:57 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:11:57.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:11:57 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:11:57.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:11:57 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.1", "format": "json"}]: dispatch 2024-09-11T08:11:57.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:11:57 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.1", "format": "json"}]: dispatch 2024-09-11T08:11:57.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:11:57 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.1", "format": "json"}]': finished 2024-09-11T08:11:57.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:11:57 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/4028291139' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:11:57.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:11:57 smithi122 ceph-mon[28992]: pgmap v145: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:11:57.986 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:11:57 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/2226302634' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-11T08:11:57.986 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:11:57 smithi122 ceph-mon[28992]: from='client.14979 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:11:57.986 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:11:57 smithi122 ceph-mon[28992]: Schedule rotate-key daemon osd.1 2024-09-11T08:11:57.986 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:11:57 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:11:57.986 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:11:57 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:11:57.986 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:11:57 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:11:57.986 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:11:57 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:11:57.986 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:11:57 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:11:57.986 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:11:57 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:11:57.986 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:11:57 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.1", "format": "json"}]: dispatch 2024-09-11T08:11:57.986 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:11:57 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.1", "format": "json"}]: dispatch 2024-09-11T08:11:57.986 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:11:57 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.1", "format": "json"}]': finished 2024-09-11T08:11:58.228 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:11:57 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: ::ffff:172.21.15.135 - - [11/Sep/2024:08:11:57] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:11:58.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:11:57 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/4028291139' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:11:58.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:11:57 smithi135 ceph-mon[32689]: pgmap v145: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:11:58.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:11:57 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/2226302634' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-11T08:11:58.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:11:57 smithi135 ceph-mon[32689]: from='client.14979 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:11:58.229 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:11:57 smithi135 ceph-mon[32689]: Schedule rotate-key daemon osd.1 2024-09-11T08:11:58.229 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:11:57 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:11:58.229 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:11:57 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:11:58.229 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:11:57 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:11:58.229 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:11:57 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:11:58.229 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:11:57 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:11:58.229 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:11:57 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:11:58.229 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:11:57 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.1", "format": "json"}]: dispatch 2024-09-11T08:11:58.229 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:11:57 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.1", "format": "json"}]: dispatch 2024-09-11T08:11:58.229 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:11:57 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.1", "format": "json"}]': finished 2024-09-11T08:11:59.095 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:11:58 smithi122 ceph-mon[39726]: Rotating authentication key for osd.1 2024-09-11T08:11:59.095 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:11:58 smithi122 ceph-mon[39726]: Reconfiguring daemon osd.1 on smithi122 2024-09-11T08:11:59.095 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:11:58 smithi122 ceph-mon[28992]: Rotating authentication key for osd.1 2024-09-11T08:11:59.096 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:11:58 smithi122 ceph-mon[28992]: Reconfiguring daemon osd.1 on smithi122 2024-09-11T08:11:59.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:11:58 smithi135 ceph-mon[32689]: Rotating authentication key for osd.1 2024-09-11T08:11:59.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:11:58 smithi135 ceph-mon[32689]: Reconfiguring daemon osd.1 on smithi122 2024-09-11T08:12:00.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:12:00 smithi135 ceph-mon[32689]: pgmap v146: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:12:00.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:12:00 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:12:00.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:12:00 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:12:00.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:12:00 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:12:00.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:12:00 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:12:00.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:12:00 smithi122 ceph-mon[39726]: pgmap v146: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:12:00.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:12:00 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:12:00.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:12:00 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:12:00.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:12:00 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:12:00.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:12:00 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:12:00.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:12:00 smithi122 ceph-mon[28992]: pgmap v146: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:12:00.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:12:00 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:12:00.735 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:12:00 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:12:00.735 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:12:00 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:12:00.735 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:12:00 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:12:02.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:12:02 smithi135 ceph-mon[32689]: pgmap v147: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:12:02.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:12:02 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:12:02.479 INFO:journalctl@ceph.iscsi.iscsi.a.smithi135.stdout:Sep 11 08:12:02 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[59799]: debug there is no tcmu-runner data available 2024-09-11T08:12:02.632 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:12:02 smithi122 ceph-mon[39726]: pgmap v147: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:12:02.633 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:12:02 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:12:02.633 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:12:02 smithi122 ceph-mon[28992]: pgmap v147: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:12:02.633 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:12:02 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:12:02.633 INFO:teuthology.orchestra.run.smithi122.stderr:++ ceph auth get-key osd.1 2024-09-11T08:12:02.949 INFO:teuthology.orchestra.run.smithi122.stderr:+ NK=AQDkTuFmacylFRAA4RuBLRnWI1X/+CLMkDzjMQ== 2024-09-11T08:12:02.949 INFO:teuthology.orchestra.run.smithi122.stderr:+ '[' AQDkTuFmacylFRAA4RuBLRnWI1X/+CLMkDzjMQ== == AQDkTuFmacylFRAA4RuBLRnWI1X/+CLMkDzjMQ== ']' 2024-09-11T08:12:02.949 INFO:teuthology.orchestra.run.smithi122.stderr:+ sleep 5 2024-09-11T08:12:03.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:12:03 smithi135 ceph-mon[32689]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:12:03.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:12:03 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/3461882473' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-11T08:12:03.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:12:03 smithi122 ceph-mon[28992]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:12:03.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:12:03 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/3461882473' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-11T08:12:03.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:12:03 smithi122 ceph-mon[39726]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:12:03.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:12:03 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/3461882473' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-11T08:12:03.734 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:12:03 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: ::ffff:172.21.15.135 - - [11/Sep/2024:08:12:03] "GET /metrics HTTP/1.1" 200 219500 "" "Prometheus/2.43.0" 2024-09-11T08:12:04.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:12:04 smithi135 ceph-mon[32689]: pgmap v148: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:12:04.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:12:04 smithi122 ceph-mon[28992]: pgmap v148: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:12:04.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:12:04 smithi122 ceph-mon[39726]: pgmap v148: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:12:06.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:12:06 smithi135 ceph-mon[32689]: pgmap v149: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:12:06.733 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:12:06 smithi122 ceph-mon[28992]: pgmap v149: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:12:06.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:12:06 smithi122 ceph-mon[39726]: pgmap v149: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:12:07.951 INFO:teuthology.orchestra.run.smithi122.stderr:++ ceph auth get-key osd.1 2024-09-11T08:12:08.228 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:12:07 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: ::ffff:172.21.15.135 - - [11/Sep/2024:08:12:07] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:12:08.265 INFO:teuthology.orchestra.run.smithi122.stderr:+ NK=AQDkTuFmacylFRAA4RuBLRnWI1X/+CLMkDzjMQ== 2024-09-11T08:12:08.265 INFO:teuthology.orchestra.run.smithi122.stderr:+ '[' AQDkTuFmacylFRAA4RuBLRnWI1X/+CLMkDzjMQ== == AQDkTuFmacylFRAA4RuBLRnWI1X/+CLMkDzjMQ== ']' 2024-09-11T08:12:08.265 INFO:teuthology.orchestra.run.smithi122.stderr:+ sleep 5 2024-09-11T08:12:08.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:12:08 smithi135 ceph-mon[32689]: pgmap v150: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:12:08.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:12:08 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/299015053' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-11T08:12:08.733 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:12:08 smithi122 ceph-mon[28992]: pgmap v150: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:12:08.733 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:12:08 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/299015053' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-11T08:12:08.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:12:08 smithi122 ceph-mon[39726]: pgmap v150: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:12:08.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:12:08 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/299015053' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-11T08:12:10.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:12:10 smithi135 ceph-mon[32689]: pgmap v151: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:12:10.733 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:12:10 smithi122 ceph-mon[28992]: pgmap v151: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:12:10.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:12:10 smithi122 ceph-mon[39726]: pgmap v151: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:12:12.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:12:12 smithi135 ceph-mon[32689]: pgmap v152: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:12:12.478 INFO:journalctl@ceph.iscsi.iscsi.a.smithi135.stdout:Sep 11 08:12:12 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[59799]: debug there is no tcmu-runner data available 2024-09-11T08:12:12.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:12:12 smithi122 ceph-mon[28992]: pgmap v152: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:12:12.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:12:12 smithi122 ceph-mon[39726]: pgmap v152: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:12:13.266 INFO:teuthology.orchestra.run.smithi122.stderr:++ ceph auth get-key osd.1 2024-09-11T08:12:13.580 INFO:teuthology.orchestra.run.smithi122.stderr:+ NK=AQDkTuFmacylFRAA4RuBLRnWI1X/+CLMkDzjMQ== 2024-09-11T08:12:13.580 INFO:teuthology.orchestra.run.smithi122.stderr:+ '[' AQDkTuFmacylFRAA4RuBLRnWI1X/+CLMkDzjMQ== == AQDkTuFmacylFRAA4RuBLRnWI1X/+CLMkDzjMQ== ']' 2024-09-11T08:12:13.580 INFO:teuthology.orchestra.run.smithi122.stderr:+ sleep 5 2024-09-11T08:12:13.580 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:12:13 smithi122 ceph-mon[39726]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:12:13.581 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:12:13 smithi122 ceph-mon[28992]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:12:13.581 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:12:13 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: ::ffff:172.21.15.135 - - [11/Sep/2024:08:12:13] "GET /metrics HTTP/1.1" 200 219500 "" "Prometheus/2.43.0" 2024-09-11T08:12:13.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:12:13 smithi135 ceph-mon[32689]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:12:14.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:12:14 smithi135 ceph-mon[32689]: pgmap v153: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:12:14.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:12:14 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/1768265336' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-11T08:12:14.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:12:14 smithi122 ceph-mon[28992]: pgmap v153: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:12:14.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:12:14 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/1768265336' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-11T08:12:14.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:12:14 smithi122 ceph-mon[39726]: pgmap v153: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:12:14.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:12:14 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/1768265336' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-11T08:12:16.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:12:16 smithi135 ceph-mon[32689]: pgmap v154: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:12:16.733 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:12:16 smithi122 ceph-mon[28992]: pgmap v154: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:12:16.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:12:16 smithi122 ceph-mon[39726]: pgmap v154: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:12:17.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:12:17 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:12:17.733 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:12:17 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:12:17.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:12:17 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:12:18.228 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:12:17 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: ::ffff:172.21.15.135 - - [11/Sep/2024:08:12:17] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:12:18.582 INFO:teuthology.orchestra.run.smithi122.stderr:++ ceph auth get-key osd.1 2024-09-11T08:12:18.582 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:12:18 smithi122 ceph-mon[28992]: pgmap v155: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:12:18.583 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:12:18 smithi122 ceph-mon[39726]: pgmap v155: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:12:18.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:12:18 smithi135 ceph-mon[32689]: pgmap v155: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:12:18.891 INFO:teuthology.orchestra.run.smithi122.stderr:+ NK=AQDkTuFmacylFRAA4RuBLRnWI1X/+CLMkDzjMQ== 2024-09-11T08:12:18.891 INFO:teuthology.orchestra.run.smithi122.stderr:+ '[' AQDkTuFmacylFRAA4RuBLRnWI1X/+CLMkDzjMQ== == AQDkTuFmacylFRAA4RuBLRnWI1X/+CLMkDzjMQ== ']' 2024-09-11T08:12:18.891 INFO:teuthology.orchestra.run.smithi122.stderr:+ sleep 5 2024-09-11T08:12:19.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:12:19 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/1012175892' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-11T08:12:19.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:12:19 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/1012175892' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-11T08:12:19.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:12:19 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/1012175892' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-11T08:12:20.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:12:20 smithi135 ceph-mon[32689]: pgmap v156: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:12:20.733 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:12:20 smithi122 ceph-mon[39726]: pgmap v156: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:12:20.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:12:20 smithi122 ceph-mon[28992]: pgmap v156: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:12:22.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:12:22 smithi135 ceph-mon[32689]: pgmap v157: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:12:22.478 INFO:journalctl@ceph.iscsi.iscsi.a.smithi135.stdout:Sep 11 08:12:22 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[59799]: debug there is no tcmu-runner data available 2024-09-11T08:12:22.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:12:22 smithi122 ceph-mon[39726]: pgmap v157: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:12:22.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:12:22 smithi122 ceph-mon[28992]: pgmap v157: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:12:23.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:12:23 smithi135 ceph-mon[32689]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:12:23.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:12:23 smithi122 ceph-mon[39726]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:12:23.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:12:23 smithi122 ceph-mon[28992]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:12:23.734 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:12:23 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: ::ffff:172.21.15.135 - - [11/Sep/2024:08:12:23] "GET /metrics HTTP/1.1" 200 219498 "" "Prometheus/2.43.0" 2024-09-11T08:12:23.892 INFO:teuthology.orchestra.run.smithi122.stderr:++ ceph auth get-key osd.1 2024-09-11T08:12:24.202 INFO:teuthology.orchestra.run.smithi122.stdout:rotating key for osd.2 2024-09-11T08:12:24.203 INFO:teuthology.orchestra.run.smithi122.stderr:+ NK=AQDNUOFmqrimKRAAb7T7SQ56IHD0DQ8SFxyKhA== 2024-09-11T08:12:24.203 INFO:teuthology.orchestra.run.smithi122.stderr:+ '[' AQDkTuFmacylFRAA4RuBLRnWI1X/+CLMkDzjMQ== == AQDNUOFmqrimKRAAb7T7SQ56IHD0DQ8SFxyKhA== ']' 2024-09-11T08:12:24.203 INFO:teuthology.orchestra.run.smithi122.stderr:+ for f in osd.0 osd.1 osd.2 osd.3 osd.4 osd.5 osd.6 osd.7 mgr.y mgr.x 2024-09-11T08:12:24.203 INFO:teuthology.orchestra.run.smithi122.stderr:+ echo 'rotating key for osd.2' 2024-09-11T08:12:24.203 INFO:teuthology.orchestra.run.smithi122.stderr:++ ceph auth get-key osd.2 2024-09-11T08:12:24.483 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:12:24 smithi122 ceph-mon[39726]: pgmap v158: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:12:24.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:12:24 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/2714406337' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-11T08:12:24.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:12:24 smithi122 ceph-mon[28992]: pgmap v158: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:12:24.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:12:24 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/2714406337' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-11T08:12:24.506 INFO:teuthology.orchestra.run.smithi122.stderr:+ K=AQD7TuFmO+D4FhAAZWMOP93t2xo7pxQkUaT2mg== 2024-09-11T08:12:24.506 INFO:teuthology.orchestra.run.smithi122.stderr:+ NK=AQD7TuFmO+D4FhAAZWMOP93t2xo7pxQkUaT2mg== 2024-09-11T08:12:24.506 INFO:teuthology.orchestra.run.smithi122.stderr:+ ceph orch daemon rotate-key osd.2 2024-09-11T08:12:24.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:12:24 smithi135 ceph-mon[32689]: pgmap v158: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:12:24.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:12:24 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/2714406337' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-11T08:12:24.760 INFO:teuthology.orchestra.run.smithi122.stdout:Scheduled to rotate-key osd.2 on host 'smithi122' 2024-09-11T08:12:24.772 INFO:teuthology.orchestra.run.smithi122.stderr:+ '[' AQD7TuFmO+D4FhAAZWMOP93t2xo7pxQkUaT2mg== == AQD7TuFmO+D4FhAAZWMOP93t2xo7pxQkUaT2mg== ']' 2024-09-11T08:12:24.772 INFO:teuthology.orchestra.run.smithi122.stderr:+ sleep 5 2024-09-11T08:12:25.592 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:12:25 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/1366065219' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-11T08:12:25.592 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:12:25 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:12:25.592 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:12:25 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:12:25.592 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:12:25 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:12:25.592 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:12:25 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:12:25.592 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:12:25 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:12:25.593 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:12:25 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:12:25.593 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:12:25 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:12:25.593 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:12:25 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:12:25.593 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:12:25 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.2", "format": "json"}]: dispatch 2024-09-11T08:12:25.593 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:12:25 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.2", "format": "json"}]: dispatch 2024-09-11T08:12:25.593 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:12:25 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.2", "format": "json"}]': finished 2024-09-11T08:12:25.593 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:12:25 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/1366065219' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-11T08:12:25.593 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:12:25 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:12:25.593 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:12:25 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:12:25.593 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:12:25 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:12:25.593 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:12:25 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:12:25.594 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:12:25 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:12:25.594 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:12:25 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:12:25.594 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:12:25 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:12:25.594 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:12:25 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:12:25.594 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:12:25 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.2", "format": "json"}]: dispatch 2024-09-11T08:12:25.594 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:12:25 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.2", "format": "json"}]: dispatch 2024-09-11T08:12:25.594 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:12:25 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.2", "format": "json"}]': finished 2024-09-11T08:12:25.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:12:25 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/1366065219' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-11T08:12:25.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:12:25 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:12:25.729 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:12:25 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:12:25.729 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:12:25 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:12:25.729 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:12:25 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:12:25.729 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:12:25 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:12:25.729 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:12:25 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:12:25.729 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:12:25 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:12:25.730 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:12:25 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:12:25.730 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:12:25 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.2", "format": "json"}]: dispatch 2024-09-11T08:12:25.730 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:12:25 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.2", "format": "json"}]: dispatch 2024-09-11T08:12:25.730 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:12:25 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.2", "format": "json"}]': finished 2024-09-11T08:12:26.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:12:26 smithi135 ceph-mon[32689]: from='client.15033 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:12:26.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:12:26 smithi135 ceph-mon[32689]: Schedule rotate-key daemon osd.2 2024-09-11T08:12:26.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:12:26 smithi135 ceph-mon[32689]: Rotating authentication key for osd.2 2024-09-11T08:12:26.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:12:26 smithi135 ceph-mon[32689]: Reconfiguring daemon osd.2 on smithi122 2024-09-11T08:12:26.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:12:26 smithi135 ceph-mon[32689]: pgmap v159: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:12:26.733 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:12:26 smithi122 ceph-mon[28992]: from='client.15033 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:12:26.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:12:26 smithi122 ceph-mon[28992]: Schedule rotate-key daemon osd.2 2024-09-11T08:12:26.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:12:26 smithi122 ceph-mon[28992]: Rotating authentication key for osd.2 2024-09-11T08:12:26.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:12:26 smithi122 ceph-mon[28992]: Reconfiguring daemon osd.2 on smithi122 2024-09-11T08:12:26.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:12:26 smithi122 ceph-mon[28992]: pgmap v159: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:12:26.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:12:26 smithi122 ceph-mon[39726]: from='client.15033 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:12:26.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:12:26 smithi122 ceph-mon[39726]: Schedule rotate-key daemon osd.2 2024-09-11T08:12:26.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:12:26 smithi122 ceph-mon[39726]: Rotating authentication key for osd.2 2024-09-11T08:12:26.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:12:26 smithi122 ceph-mon[39726]: Reconfiguring daemon osd.2 on smithi122 2024-09-11T08:12:26.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:12:26 smithi122 ceph-mon[39726]: pgmap v159: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:12:27.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:12:27 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:12:27.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:12:27 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:12:27.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:12:27 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:12:27.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:12:27 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:12:27.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:12:27 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:12:27.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:12:27 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:12:27.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:12:27 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:12:27.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:12:27 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:12:27.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:12:27 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:12:27.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:12:27 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:12:27.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:12:27 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:12:27.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:12:27 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:12:28.228 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:12:27 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: ::ffff:172.21.15.135 - - [11/Sep/2024:08:12:27] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:12:28.727 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:12:28 smithi135 ceph-mon[32689]: pgmap v160: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:12:28.733 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:12:28 smithi122 ceph-mon[28992]: pgmap v160: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:12:28.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:12:28 smithi122 ceph-mon[39726]: pgmap v160: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:12:29.774 INFO:teuthology.orchestra.run.smithi122.stderr:++ ceph auth get-key osd.2 2024-09-11T08:12:30.078 INFO:teuthology.orchestra.run.smithi122.stderr:+ NK=AQD7TuFmO+D4FhAAZWMOP93t2xo7pxQkUaT2mg== 2024-09-11T08:12:30.078 INFO:teuthology.orchestra.run.smithi122.stderr:+ '[' AQD7TuFmO+D4FhAAZWMOP93t2xo7pxQkUaT2mg== == AQD7TuFmO+D4FhAAZWMOP93t2xo7pxQkUaT2mg== ']' 2024-09-11T08:12:30.078 INFO:teuthology.orchestra.run.smithi122.stderr:+ sleep 5 2024-09-11T08:12:30.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:12:30 smithi135 ceph-mon[32689]: pgmap v161: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:12:30.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:12:30 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/2500981170' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-11T08:12:30.733 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:12:30 smithi122 ceph-mon[28992]: pgmap v161: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:12:30.733 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:12:30 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/2500981170' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-11T08:12:30.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:12:30 smithi122 ceph-mon[39726]: pgmap v161: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:12:30.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:12:30 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/2500981170' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-11T08:12:32.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:12:32 smithi135 ceph-mon[32689]: pgmap v162: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:12:32.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:12:32 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:12:32.478 INFO:journalctl@ceph.iscsi.iscsi.a.smithi135.stdout:Sep 11 08:12:32 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[59799]: debug there is no tcmu-runner data available 2024-09-11T08:12:32.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:12:32 smithi122 ceph-mon[28992]: pgmap v162: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:12:32.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:12:32 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:12:32.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:12:32 smithi122 ceph-mon[39726]: pgmap v162: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:12:32.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:12:32 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:12:33.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:12:33 smithi135 ceph-mon[32689]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:12:33.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:12:33 smithi122 ceph-mon[39726]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:12:33.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:12:33 smithi122 ceph-mon[28992]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:12:33.734 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:12:33 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: ::ffff:172.21.15.135 - - [11/Sep/2024:08:12:33] "GET /metrics HTTP/1.1" 200 219534 "" "Prometheus/2.43.0" 2024-09-11T08:12:34.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:12:34 smithi135 ceph-mon[32689]: pgmap v163: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:12:34.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:12:34 smithi122 ceph-mon[28992]: pgmap v163: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:12:34.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:12:34 smithi122 ceph-mon[39726]: pgmap v163: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:12:35.080 INFO:teuthology.orchestra.run.smithi122.stderr:++ ceph auth get-key osd.2 2024-09-11T08:12:35.392 INFO:teuthology.orchestra.run.smithi122.stderr:+ NK=AQD7TuFmO+D4FhAAZWMOP93t2xo7pxQkUaT2mg== 2024-09-11T08:12:35.392 INFO:teuthology.orchestra.run.smithi122.stderr:+ '[' AQD7TuFmO+D4FhAAZWMOP93t2xo7pxQkUaT2mg== == AQD7TuFmO+D4FhAAZWMOP93t2xo7pxQkUaT2mg== ']' 2024-09-11T08:12:35.392 INFO:teuthology.orchestra.run.smithi122.stderr:+ sleep 5 2024-09-11T08:12:35.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:12:35 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/653737454' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-11T08:12:35.733 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:12:35 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/653737454' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-11T08:12:35.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:12:35 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/653737454' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-11T08:12:36.727 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:12:36 smithi135 ceph-mon[32689]: pgmap v164: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:12:36.733 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:12:36 smithi122 ceph-mon[28992]: pgmap v164: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:12:36.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:12:36 smithi122 ceph-mon[39726]: pgmap v164: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:12:38.228 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:12:37 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: ::ffff:172.21.15.135 - - [11/Sep/2024:08:12:37] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:12:38.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:12:38 smithi135 ceph-mon[32689]: pgmap v165: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:12:38.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:12:38 smithi122 ceph-mon[28992]: pgmap v165: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:12:38.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:12:38 smithi122 ceph-mon[39726]: pgmap v165: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:12:40.392 INFO:teuthology.orchestra.run.smithi122.stderr:++ ceph auth get-key osd.2 2024-09-11T08:12:40.706 INFO:teuthology.orchestra.run.smithi122.stderr:+ NK=AQD7TuFmO+D4FhAAZWMOP93t2xo7pxQkUaT2mg== 2024-09-11T08:12:40.706 INFO:teuthology.orchestra.run.smithi122.stderr:+ '[' AQD7TuFmO+D4FhAAZWMOP93t2xo7pxQkUaT2mg== == AQD7TuFmO+D4FhAAZWMOP93t2xo7pxQkUaT2mg== ']' 2024-09-11T08:12:40.707 INFO:teuthology.orchestra.run.smithi122.stderr:+ sleep 5 2024-09-11T08:12:40.707 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:12:40 smithi122 ceph-mon[39726]: pgmap v166: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:12:40.707 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:12:40 smithi122 ceph-mon[28992]: pgmap v166: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:12:40.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:12:40 smithi135 ceph-mon[32689]: pgmap v166: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:12:41.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:12:41 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/3356729978' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-11T08:12:41.733 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:12:41 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/3356729978' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-11T08:12:41.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:12:41 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/3356729978' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-11T08:12:42.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:12:42 smithi135 ceph-mon[32689]: pgmap v167: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:12:42.728 INFO:journalctl@ceph.iscsi.iscsi.a.smithi135.stdout:Sep 11 08:12:42 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[59799]: debug there is no tcmu-runner data available 2024-09-11T08:12:42.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:12:42 smithi122 ceph-mon[39726]: pgmap v167: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:12:42.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:12:42 smithi122 ceph-mon[28992]: pgmap v167: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:12:43.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:12:43 smithi122 ceph-mon[28992]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:12:43.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:12:43 smithi122 ceph-mon[39726]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:12:43.734 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:12:43 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: ::ffff:172.21.15.135 - - [11/Sep/2024:08:12:43] "GET /metrics HTTP/1.1" 200 219534 "" "Prometheus/2.43.0" 2024-09-11T08:12:43.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:12:43 smithi135 ceph-mon[32689]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:12:44.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:12:44 smithi122 ceph-mon[28992]: pgmap v168: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:12:44.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:12:44 smithi122 ceph-mon[39726]: pgmap v168: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:12:44.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:12:44 smithi135 ceph-mon[32689]: pgmap v168: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:12:45.708 INFO:teuthology.orchestra.run.smithi122.stderr:++ ceph auth get-key osd.2 2024-09-11T08:12:46.022 INFO:teuthology.orchestra.run.smithi122.stdout:rotating key for osd.3 2024-09-11T08:12:46.022 INFO:teuthology.orchestra.run.smithi122.stderr:+ NK=AQDpUOFmKekXBxAAW0Be1449JSzbKd7BSW2mJw== 2024-09-11T08:12:46.023 INFO:teuthology.orchestra.run.smithi122.stderr:+ '[' AQD7TuFmO+D4FhAAZWMOP93t2xo7pxQkUaT2mg== == AQDpUOFmKekXBxAAW0Be1449JSzbKd7BSW2mJw== ']' 2024-09-11T08:12:46.023 INFO:teuthology.orchestra.run.smithi122.stderr:+ for f in osd.0 osd.1 osd.2 osd.3 osd.4 osd.5 osd.6 osd.7 mgr.y mgr.x 2024-09-11T08:12:46.023 INFO:teuthology.orchestra.run.smithi122.stderr:+ echo 'rotating key for osd.3' 2024-09-11T08:12:46.023 INFO:teuthology.orchestra.run.smithi122.stderr:++ ceph auth get-key osd.3 2024-09-11T08:12:46.325 INFO:teuthology.orchestra.run.smithi122.stderr:+ K=AQATT+FmbYU/IBAAlCykbaAYtkLwbw8S0XsIpQ== 2024-09-11T08:12:46.325 INFO:teuthology.orchestra.run.smithi122.stderr:+ NK=AQATT+FmbYU/IBAAlCykbaAYtkLwbw8S0XsIpQ== 2024-09-11T08:12:46.325 INFO:teuthology.orchestra.run.smithi122.stderr:+ ceph orch daemon rotate-key osd.3 2024-09-11T08:12:46.589 INFO:teuthology.orchestra.run.smithi122.stdout:Scheduled to rotate-key osd.3 on host 'smithi122' 2024-09-11T08:12:46.590 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:12:46 smithi122 ceph-mon[28992]: pgmap v169: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:12:46.590 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:12:46 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/3438892028' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-11T08:12:46.590 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:12:46 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/2960658276' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-11T08:12:46.590 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:12:46 smithi122 ceph-mon[39726]: pgmap v169: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:12:46.590 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:12:46 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/3438892028' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-11T08:12:46.590 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:12:46 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/2960658276' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-11T08:12:46.601 INFO:teuthology.orchestra.run.smithi122.stderr:+ '[' AQATT+FmbYU/IBAAlCykbaAYtkLwbw8S0XsIpQ== == AQATT+FmbYU/IBAAlCykbaAYtkLwbw8S0XsIpQ== ']' 2024-09-11T08:12:46.601 INFO:teuthology.orchestra.run.smithi122.stderr:+ sleep 5 2024-09-11T08:12:46.977 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:12:46 smithi135 ceph-mon[32689]: pgmap v169: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:12:46.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:12:46 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/3438892028' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-11T08:12:46.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:12:46 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/2960658276' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-11T08:12:47.935 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:12:47 smithi135 ceph-mon[32689]: from='client.15081 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:12:47.936 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:12:47 smithi135 ceph-mon[32689]: Schedule rotate-key daemon osd.3 2024-09-11T08:12:47.936 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:12:47 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:12:47.936 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:12:47 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:12:47.936 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:12:47 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:12:47.936 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:12:47 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:12:47.936 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:12:47 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:12:47.936 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:12:47 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:12:47.936 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:12:47 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:12:47.936 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:12:47 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:12:47.936 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:12:47 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.3", "format": "json"}]: dispatch 2024-09-11T08:12:47.936 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:12:47 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.3", "format": "json"}]: dispatch 2024-09-11T08:12:47.936 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:12:47 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.3", "format": "json"}]': finished 2024-09-11T08:12:47.937 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:12:47 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:12:47.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:12:47 smithi122 ceph-mon[39726]: from='client.15081 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:12:47.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:12:47 smithi122 ceph-mon[39726]: Schedule rotate-key daemon osd.3 2024-09-11T08:12:47.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:12:47 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:12:47.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:12:47 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:12:47.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:12:47 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:12:47.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:12:47 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:12:47.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:12:47 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:12:47.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:12:47 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:12:47.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:12:47 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:12:47.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:12:47 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:12:47.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:12:47 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.3", "format": "json"}]: dispatch 2024-09-11T08:12:47.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:12:47 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.3", "format": "json"}]: dispatch 2024-09-11T08:12:47.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:12:47 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.3", "format": "json"}]': finished 2024-09-11T08:12:47.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:12:47 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:12:47.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:12:47 smithi122 ceph-mon[28992]: from='client.15081 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:12:47.986 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:12:47 smithi122 ceph-mon[28992]: Schedule rotate-key daemon osd.3 2024-09-11T08:12:47.986 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:12:47 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:12:47.986 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:12:47 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:12:47.986 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:12:47 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:12:47.986 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:12:47 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:12:47.986 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:12:47 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:12:47.986 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:12:47 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:12:47.986 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:12:47 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:12:47.986 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:12:47 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:12:47.986 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:12:47 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.3", "format": "json"}]: dispatch 2024-09-11T08:12:47.986 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:12:47 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.3", "format": "json"}]: dispatch 2024-09-11T08:12:47.986 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:12:47 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.3", "format": "json"}]': finished 2024-09-11T08:12:47.986 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:12:47 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:12:48.228 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:12:47 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: ::ffff:172.21.15.135 - - [11/Sep/2024:08:12:47] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:12:48.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:12:48 smithi122 ceph-mon[28992]: Rotating authentication key for osd.3 2024-09-11T08:12:48.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:12:48 smithi122 ceph-mon[28992]: Reconfiguring daemon osd.3 on smithi122 2024-09-11T08:12:48.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:12:48 smithi122 ceph-mon[28992]: pgmap v170: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:12:48.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:12:48 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:12:48.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:12:48 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:12:48.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:12:48 smithi122 ceph-mon[39726]: Rotating authentication key for osd.3 2024-09-11T08:12:48.735 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:12:48 smithi122 ceph-mon[39726]: Reconfiguring daemon osd.3 on smithi122 2024-09-11T08:12:48.735 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:12:48 smithi122 ceph-mon[39726]: pgmap v170: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:12:48.735 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:12:48 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:12:48.735 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:12:48 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:12:48.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:12:48 smithi135 ceph-mon[32689]: Rotating authentication key for osd.3 2024-09-11T08:12:48.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:12:48 smithi135 ceph-mon[32689]: Reconfiguring daemon osd.3 on smithi122 2024-09-11T08:12:48.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:12:48 smithi135 ceph-mon[32689]: pgmap v170: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:12:48.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:12:48 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:12:48.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:12:48 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:12:49.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:12:49 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:12:49.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:12:49 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:12:49.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:12:49 smithi135 ceph-mon[32689]: pgmap v171: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:12:49.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:12:49 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:12:49.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:12:49 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:12:49.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:12:49 smithi122 ceph-mon[39726]: pgmap v171: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:12:49.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:12:49 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:12:49.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:12:49 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:12:49.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:12:49 smithi122 ceph-mon[28992]: pgmap v171: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:12:51.604 INFO:teuthology.orchestra.run.smithi122.stderr:++ ceph auth get-key osd.3 2024-09-11T08:12:51.914 INFO:teuthology.orchestra.run.smithi122.stderr:+ NK=AQATT+FmbYU/IBAAlCykbaAYtkLwbw8S0XsIpQ== 2024-09-11T08:12:51.914 INFO:teuthology.orchestra.run.smithi122.stderr:+ '[' AQATT+FmbYU/IBAAlCykbaAYtkLwbw8S0XsIpQ== == AQATT+FmbYU/IBAAlCykbaAYtkLwbw8S0XsIpQ== ']' 2024-09-11T08:12:51.914 INFO:teuthology.orchestra.run.smithi122.stderr:+ sleep 5 2024-09-11T08:12:52.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:12:52 smithi135 ceph-mon[32689]: pgmap v172: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:12:52.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:12:52 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/4218254937' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-11T08:12:52.478 INFO:journalctl@ceph.iscsi.iscsi.a.smithi135.stdout:Sep 11 08:12:52 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[59799]: debug there is no tcmu-runner data available 2024-09-11T08:12:52.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:12:52 smithi122 ceph-mon[39726]: pgmap v172: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:12:52.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:12:52 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/4218254937' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-11T08:12:52.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:12:52 smithi122 ceph-mon[28992]: pgmap v172: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:12:52.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:12:52 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/4218254937' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-11T08:12:53.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:12:53 smithi135 ceph-mon[32689]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:12:53.518 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:12:53 smithi122 ceph-mon[28992]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:12:53.518 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:12:53 smithi122 ceph-mon[39726]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:12:53.518 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:12:53 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: ::ffff:172.21.15.135 - - [11/Sep/2024:08:12:53] "GET /metrics HTTP/1.1" 200 219537 "" "Prometheus/2.43.0" 2024-09-11T08:12:54.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:12:54 smithi135 ceph-mon[32689]: pgmap v173: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:12:54.733 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:12:54 smithi122 ceph-mon[28992]: pgmap v173: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:12:54.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:12:54 smithi122 ceph-mon[39726]: pgmap v173: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:12:56.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:12:56 smithi135 ceph-mon[32689]: pgmap v174: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:12:56.733 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:12:56 smithi122 ceph-mon[28992]: pgmap v174: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:12:56.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:12:56 smithi122 ceph-mon[39726]: pgmap v174: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:12:56.916 INFO:teuthology.orchestra.run.smithi122.stderr:++ ceph auth get-key osd.3 2024-09-11T08:12:57.226 INFO:teuthology.orchestra.run.smithi122.stderr:+ NK=AQATT+FmbYU/IBAAlCykbaAYtkLwbw8S0XsIpQ== 2024-09-11T08:12:57.226 INFO:teuthology.orchestra.run.smithi122.stderr:+ '[' AQATT+FmbYU/IBAAlCykbaAYtkLwbw8S0XsIpQ== == AQATT+FmbYU/IBAAlCykbaAYtkLwbw8S0XsIpQ== ']' 2024-09-11T08:12:57.226 INFO:teuthology.orchestra.run.smithi122.stderr:+ sleep 5 2024-09-11T08:12:57.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:12:57 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/2170073478' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-11T08:12:57.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:12:57 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/2170073478' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-11T08:12:57.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:12:57 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/2170073478' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-11T08:12:58.228 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:12:57 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: ::ffff:172.21.15.135 - - [11/Sep/2024:08:12:57] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:12:58.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:12:58 smithi135 ceph-mon[32689]: pgmap v175: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:12:58.733 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:12:58 smithi122 ceph-mon[28992]: pgmap v175: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:12:58.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:12:58 smithi122 ceph-mon[39726]: pgmap v175: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:13:00.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:00 smithi135 ceph-mon[32689]: pgmap v176: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:13:00.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:00 smithi122 ceph-mon[28992]: pgmap v176: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:13:00.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:00 smithi122 ceph-mon[39726]: pgmap v176: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:13:02.229 INFO:teuthology.orchestra.run.smithi122.stderr:++ ceph auth get-key osd.3 2024-09-11T08:13:02.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:02 smithi122 ceph-mon[28992]: pgmap v177: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:13:02.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:02 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:13:02.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:02 smithi122 ceph-mon[39726]: pgmap v177: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:13:02.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:02 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:13:02.543 INFO:teuthology.orchestra.run.smithi122.stderr:+ NK=AQATT+FmbYU/IBAAlCykbaAYtkLwbw8S0XsIpQ== 2024-09-11T08:13:02.543 INFO:teuthology.orchestra.run.smithi122.stderr:+ '[' AQATT+FmbYU/IBAAlCykbaAYtkLwbw8S0XsIpQ== == AQATT+FmbYU/IBAAlCykbaAYtkLwbw8S0XsIpQ== ']' 2024-09-11T08:13:02.543 INFO:teuthology.orchestra.run.smithi122.stderr:+ sleep 5 2024-09-11T08:13:02.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:02 smithi135 ceph-mon[32689]: pgmap v177: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:13:02.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:02 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:13:02.728 INFO:journalctl@ceph.iscsi.iscsi.a.smithi135.stdout:Sep 11 08:13:02 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[59799]: debug there is no tcmu-runner data available 2024-09-11T08:13:03.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:03 smithi135 ceph-mon[32689]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:13:03.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:03 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/1955000085' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-11T08:13:03.734 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:13:03 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: ::ffff:172.21.15.135 - - [11/Sep/2024:08:13:03] "GET /metrics HTTP/1.1" 200 219528 "" "Prometheus/2.43.0" 2024-09-11T08:13:03.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:03 smithi122 ceph-mon[28992]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:13:03.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:03 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/1955000085' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-11T08:13:03.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:03 smithi122 ceph-mon[39726]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:13:03.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:03 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/1955000085' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-11T08:13:04.729 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:04 smithi135 ceph-mon[32689]: pgmap v178: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:13:04.733 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:04 smithi122 ceph-mon[28992]: pgmap v178: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:13:04.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:04 smithi122 ceph-mon[39726]: pgmap v178: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:13:06.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:06 smithi135 ceph-mon[32689]: pgmap v179: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:13:06.733 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:06 smithi122 ceph-mon[28992]: pgmap v179: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:13:06.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:06 smithi122 ceph-mon[39726]: pgmap v179: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:13:07.545 INFO:teuthology.orchestra.run.smithi122.stderr:++ ceph auth get-key osd.3 2024-09-11T08:13:07.863 INFO:teuthology.orchestra.run.smithi122.stderr:+ NK=AQATT+FmbYU/IBAAlCykbaAYtkLwbw8S0XsIpQ== 2024-09-11T08:13:07.864 INFO:teuthology.orchestra.run.smithi122.stderr:+ '[' AQATT+FmbYU/IBAAlCykbaAYtkLwbw8S0XsIpQ== == AQATT+FmbYU/IBAAlCykbaAYtkLwbw8S0XsIpQ== ']' 2024-09-11T08:13:07.864 INFO:teuthology.orchestra.run.smithi122.stderr:+ sleep 5 2024-09-11T08:13:08.228 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:13:07 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: ::ffff:172.21.15.135 - - [11/Sep/2024:08:13:07] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:13:08.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:08 smithi135 ceph-mon[32689]: pgmap v180: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:13:08.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:08 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/1404329328' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-11T08:13:08.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:08 smithi122 ceph-mon[28992]: pgmap v180: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:13:08.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:08 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/1404329328' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-11T08:13:08.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:08 smithi122 ceph-mon[39726]: pgmap v180: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:13:08.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:08 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/1404329328' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-11T08:13:10.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:10 smithi135 ceph-mon[32689]: pgmap v181: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:13:10.733 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:10 smithi122 ceph-mon[28992]: pgmap v181: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:13:10.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:10 smithi122 ceph-mon[39726]: pgmap v181: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:13:12.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:12 smithi135 ceph-mon[32689]: pgmap v182: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:13:12.728 INFO:journalctl@ceph.iscsi.iscsi.a.smithi135.stdout:Sep 11 08:13:12 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[59799]: debug there is no tcmu-runner data available 2024-09-11T08:13:12.733 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:12 smithi122 ceph-mon[28992]: pgmap v182: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:13:12.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:12 smithi122 ceph-mon[39726]: pgmap v182: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:13:12.865 INFO:teuthology.orchestra.run.smithi122.stderr:++ ceph auth get-key osd.3 2024-09-11T08:13:13.181 INFO:teuthology.orchestra.run.smithi122.stdout:rotating key for osd.4 2024-09-11T08:13:13.181 INFO:teuthology.orchestra.run.smithi122.stderr:+ NK=AQD+UOFmx4byOBAAgBZTN8UAYWuw0qn9xQ8aXg== 2024-09-11T08:13:13.181 INFO:teuthology.orchestra.run.smithi122.stderr:+ '[' AQATT+FmbYU/IBAAlCykbaAYtkLwbw8S0XsIpQ== == AQD+UOFmx4byOBAAgBZTN8UAYWuw0qn9xQ8aXg== ']' 2024-09-11T08:13:13.181 INFO:teuthology.orchestra.run.smithi122.stderr:+ for f in osd.0 osd.1 osd.2 osd.3 osd.4 osd.5 osd.6 osd.7 mgr.y mgr.x 2024-09-11T08:13:13.181 INFO:teuthology.orchestra.run.smithi122.stderr:+ echo 'rotating key for osd.4' 2024-09-11T08:13:13.181 INFO:teuthology.orchestra.run.smithi122.stderr:++ ceph auth get-key osd.4 2024-09-11T08:13:13.459 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:13:13 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: ::ffff:172.21.15.135 - - [11/Sep/2024:08:13:13] "GET /metrics HTTP/1.1" 200 219528 "" "Prometheus/2.43.0" 2024-09-11T08:13:13.486 INFO:teuthology.orchestra.run.smithi122.stderr:+ K=AQApT+Fmq7L9LBAAkOWWEe24ug7gk9Hy3oFatA== 2024-09-11T08:13:13.486 INFO:teuthology.orchestra.run.smithi122.stderr:+ NK=AQApT+Fmq7L9LBAAkOWWEe24ug7gk9Hy3oFatA== 2024-09-11T08:13:13.487 INFO:teuthology.orchestra.run.smithi122.stderr:+ ceph orch daemon rotate-key osd.4 2024-09-11T08:13:13.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:13 smithi135 ceph-mon[32689]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:13:13.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:13 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/1387389661' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-11T08:13:13.733 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:13 smithi122 ceph-mon[28992]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:13:13.733 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:13 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/1387389661' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-11T08:13:13.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:13 smithi122 ceph-mon[39726]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:13:13.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:13 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/1387389661' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-11T08:13:13.743 INFO:teuthology.orchestra.run.smithi122.stdout:Scheduled to rotate-key osd.4 on host 'smithi135' 2024-09-11T08:13:13.755 INFO:teuthology.orchestra.run.smithi122.stderr:+ '[' AQApT+Fmq7L9LBAAkOWWEe24ug7gk9Hy3oFatA== == AQApT+Fmq7L9LBAAkOWWEe24ug7gk9Hy3oFatA== ']' 2024-09-11T08:13:13.755 INFO:teuthology.orchestra.run.smithi122.stderr:+ sleep 5 2024-09-11T08:13:14.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:14 smithi135 ceph-mon[32689]: pgmap v183: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:13:14.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:14 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/1319050922' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-11T08:13:14.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:14 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:13:14.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:14 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:13:14.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:14 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:13:14.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:14 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:13:14.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:14 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:13:14.729 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:14 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:13:14.729 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:14 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.4", "format": "json"}]: dispatch 2024-09-11T08:13:14.729 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:14 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.4", "format": "json"}]: dispatch 2024-09-11T08:13:14.729 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:14 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.4", "format": "json"}]': finished 2024-09-11T08:13:14.733 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:14 smithi122 ceph-mon[39726]: pgmap v183: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:13:14.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:14 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/1319050922' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-11T08:13:14.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:14 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:13:14.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:14 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:13:14.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:14 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:13:14.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:14 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:13:14.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:14 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:13:14.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:14 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:13:14.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:14 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.4", "format": "json"}]: dispatch 2024-09-11T08:13:14.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:14 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.4", "format": "json"}]: dispatch 2024-09-11T08:13:14.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:14 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.4", "format": "json"}]': finished 2024-09-11T08:13:14.735 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:14 smithi122 ceph-mon[28992]: pgmap v183: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:13:14.735 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:14 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/1319050922' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-11T08:13:14.735 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:14 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:13:14.735 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:14 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:13:14.735 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:14 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:13:14.735 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:14 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:13:14.735 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:14 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:13:14.735 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:14 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:13:14.735 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:14 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.4", "format": "json"}]: dispatch 2024-09-11T08:13:14.735 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:14 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.4", "format": "json"}]: dispatch 2024-09-11T08:13:14.735 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:14 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.4", "format": "json"}]': finished 2024-09-11T08:13:15.665 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:15 smithi135 ceph-mon[32689]: from='client.15135 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:13:15.665 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:15 smithi135 ceph-mon[32689]: Schedule rotate-key daemon osd.4 2024-09-11T08:13:15.665 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:15 smithi135 ceph-mon[32689]: Rotating authentication key for osd.4 2024-09-11T08:13:15.665 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:15 smithi135 ceph-mon[32689]: Reconfiguring daemon osd.4 on smithi135 2024-09-11T08:13:15.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:15 smithi122 ceph-mon[28992]: from='client.15135 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:13:15.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:15 smithi122 ceph-mon[28992]: Schedule rotate-key daemon osd.4 2024-09-11T08:13:15.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:15 smithi122 ceph-mon[28992]: Rotating authentication key for osd.4 2024-09-11T08:13:15.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:15 smithi122 ceph-mon[28992]: Reconfiguring daemon osd.4 on smithi135 2024-09-11T08:13:15.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:15 smithi122 ceph-mon[39726]: from='client.15135 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:13:15.735 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:15 smithi122 ceph-mon[39726]: Schedule rotate-key daemon osd.4 2024-09-11T08:13:15.735 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:15 smithi122 ceph-mon[39726]: Rotating authentication key for osd.4 2024-09-11T08:13:15.735 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:15 smithi122 ceph-mon[39726]: Reconfiguring daemon osd.4 on smithi135 2024-09-11T08:13:16.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:16 smithi135 ceph-mon[32689]: pgmap v184: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:13:16.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:16 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:13:16.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:16 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:13:16.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:16 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:13:16.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:16 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:13:16.733 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:16 smithi122 ceph-mon[39726]: pgmap v184: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:13:16.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:16 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:13:16.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:16 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:13:16.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:16 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:13:16.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:16 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:13:16.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:16 smithi122 ceph-mon[28992]: pgmap v184: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:13:16.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:16 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:13:16.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:16 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:13:16.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:16 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:13:16.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:16 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:13:17.729 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:17 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:13:17.733 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:17 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:13:17.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:17 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:13:18.228 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:13:17 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: ::ffff:172.21.15.135 - - [11/Sep/2024:08:13:17] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:13:18.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:18 smithi135 ceph-mon[32689]: pgmap v185: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:13:18.733 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:18 smithi122 ceph-mon[28992]: pgmap v185: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:13:18.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:18 smithi122 ceph-mon[39726]: pgmap v185: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:13:18.757 INFO:teuthology.orchestra.run.smithi122.stderr:++ ceph auth get-key osd.4 2024-09-11T08:13:19.068 INFO:teuthology.orchestra.run.smithi122.stderr:+ NK=AQApT+Fmq7L9LBAAkOWWEe24ug7gk9Hy3oFatA== 2024-09-11T08:13:19.068 INFO:teuthology.orchestra.run.smithi122.stderr:+ '[' AQApT+Fmq7L9LBAAkOWWEe24ug7gk9Hy3oFatA== == AQApT+Fmq7L9LBAAkOWWEe24ug7gk9Hy3oFatA== ']' 2024-09-11T08:13:19.068 INFO:teuthology.orchestra.run.smithi122.stderr:+ sleep 5 2024-09-11T08:13:19.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:19 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/3347467245' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-11T08:13:19.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:19 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/3347467245' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-11T08:13:19.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:19 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/3347467245' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-11T08:13:20.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:20 smithi122 ceph-mon[39726]: pgmap v186: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:13:20.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:20 smithi122 ceph-mon[28992]: pgmap v186: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:13:20.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:20 smithi135 ceph-mon[32689]: pgmap v186: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:13:22.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:22 smithi135 ceph-mon[32689]: pgmap v187: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:13:22.728 INFO:journalctl@ceph.iscsi.iscsi.a.smithi135.stdout:Sep 11 08:13:22 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[59799]: debug there is no tcmu-runner data available 2024-09-11T08:13:22.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:22 smithi122 ceph-mon[39726]: pgmap v187: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:13:22.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:22 smithi122 ceph-mon[28992]: pgmap v187: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:13:23.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:23 smithi122 ceph-mon[28992]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:13:23.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:23 smithi122 ceph-mon[39726]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:13:23.734 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:13:23 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: ::ffff:172.21.15.135 - - [11/Sep/2024:08:13:23] "GET /metrics HTTP/1.1" 200 219523 "" "Prometheus/2.43.0" 2024-09-11T08:13:23.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:23 smithi135 ceph-mon[32689]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:13:24.070 INFO:teuthology.orchestra.run.smithi122.stderr:++ ceph auth get-key osd.4 2024-09-11T08:13:24.389 INFO:teuthology.orchestra.run.smithi122.stderr:+ NK=AQApT+Fmq7L9LBAAkOWWEe24ug7gk9Hy3oFatA== 2024-09-11T08:13:24.389 INFO:teuthology.orchestra.run.smithi122.stderr:+ '[' AQApT+Fmq7L9LBAAkOWWEe24ug7gk9Hy3oFatA== == AQApT+Fmq7L9LBAAkOWWEe24ug7gk9Hy3oFatA== ']' 2024-09-11T08:13:24.389 INFO:teuthology.orchestra.run.smithi122.stderr:+ sleep 5 2024-09-11T08:13:24.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:24 smithi122 ceph-mon[28992]: pgmap v188: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:13:24.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:24 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/131679506' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-11T08:13:24.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:24 smithi122 ceph-mon[39726]: pgmap v188: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:13:24.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:24 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/131679506' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-11T08:13:24.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:24 smithi135 ceph-mon[32689]: pgmap v188: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:13:24.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:24 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/131679506' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-11T08:13:26.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:26 smithi135 ceph-mon[32689]: pgmap v189: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:13:26.983 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:26 smithi122 ceph-mon[28992]: pgmap v189: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:13:26.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:26 smithi122 ceph-mon[39726]: pgmap v189: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:13:28.228 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:13:27 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: ::ffff:172.21.15.135 - - [11/Sep/2024:08:13:27] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:13:28.786 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:28 smithi122 ceph-mon[39726]: pgmap v190: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:13:28.786 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:28 smithi122 ceph-mon[28992]: pgmap v190: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:13:28.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:28 smithi135 ceph-mon[32689]: pgmap v190: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:13:29.390 INFO:teuthology.orchestra.run.smithi122.stderr:++ ceph auth get-key osd.4 2024-09-11T08:13:29.707 INFO:teuthology.orchestra.run.smithi122.stdout:rotating key for osd.5 2024-09-11T08:13:29.708 INFO:teuthology.orchestra.run.smithi122.stderr:+ NK=AQAaUeFmUWqWCxAAx5d5qIlFU9+geBlANFupsg== 2024-09-11T08:13:29.708 INFO:teuthology.orchestra.run.smithi122.stderr:+ '[' AQApT+Fmq7L9LBAAkOWWEe24ug7gk9Hy3oFatA== == AQAaUeFmUWqWCxAAx5d5qIlFU9+geBlANFupsg== ']' 2024-09-11T08:13:29.708 INFO:teuthology.orchestra.run.smithi122.stderr:+ for f in osd.0 osd.1 osd.2 osd.3 osd.4 osd.5 osd.6 osd.7 mgr.y mgr.x 2024-09-11T08:13:29.708 INFO:teuthology.orchestra.run.smithi122.stderr:+ echo 'rotating key for osd.5' 2024-09-11T08:13:29.708 INFO:teuthology.orchestra.run.smithi122.stderr:++ ceph auth get-key osd.5 2024-09-11T08:13:29.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:29 smithi135 ceph-mon[32689]: pgmap v191: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:13:29.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:29 smithi122 ceph-mon[28992]: pgmap v191: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:13:29.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:29 smithi122 ceph-mon[39726]: pgmap v191: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:13:30.020 INFO:teuthology.orchestra.run.smithi122.stderr:+ K=AQBBT+Fm7GU/IxAAolcMJmBKrSHvvukDJ475iw== 2024-09-11T08:13:30.021 INFO:teuthology.orchestra.run.smithi122.stderr:+ NK=AQBBT+Fm7GU/IxAAolcMJmBKrSHvvukDJ475iw== 2024-09-11T08:13:30.021 INFO:teuthology.orchestra.run.smithi122.stderr:+ ceph orch daemon rotate-key osd.5 2024-09-11T08:13:30.289 INFO:teuthology.orchestra.run.smithi122.stdout:Scheduled to rotate-key osd.5 on host 'smithi135' 2024-09-11T08:13:30.304 INFO:teuthology.orchestra.run.smithi122.stderr:+ '[' AQBBT+Fm7GU/IxAAolcMJmBKrSHvvukDJ475iw== == AQBBT+Fm7GU/IxAAolcMJmBKrSHvvukDJ475iw== ']' 2024-09-11T08:13:30.304 INFO:teuthology.orchestra.run.smithi122.stderr:+ sleep 5 2024-09-11T08:13:30.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:30 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/2965468748' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-11T08:13:30.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:30 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/3187313545' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-11T08:13:30.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:30 smithi135 ceph-mon[32689]: from='client.15177 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.5", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:13:30.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:30 smithi135 ceph-mon[32689]: Schedule rotate-key daemon osd.5 2024-09-11T08:13:30.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:30 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:13:30.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:30 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:13:30.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:30 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:13:30.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:30 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:13:30.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:30 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:13:30.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:30 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:13:30.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:30 smithi135 ceph-mon[32689]: Rotating authentication key for osd.5 2024-09-11T08:13:30.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:30 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.5", "format": "json"}]: dispatch 2024-09-11T08:13:30.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:30 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.5", "format": "json"}]: dispatch 2024-09-11T08:13:30.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:30 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.5", "format": "json"}]': finished 2024-09-11T08:13:30.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:30 smithi135 ceph-mon[32689]: Reconfiguring daemon osd.5 on smithi135 2024-09-11T08:13:30.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:30 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/2965468748' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-11T08:13:30.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:30 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/3187313545' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-11T08:13:30.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:30 smithi122 ceph-mon[28992]: from='client.15177 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.5", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:13:30.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:30 smithi122 ceph-mon[28992]: Schedule rotate-key daemon osd.5 2024-09-11T08:13:30.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:30 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:13:30.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:30 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:13:30.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:30 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:13:30.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:30 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:13:30.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:30 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:13:30.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:30 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:13:30.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:30 smithi122 ceph-mon[28992]: Rotating authentication key for osd.5 2024-09-11T08:13:30.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:30 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.5", "format": "json"}]: dispatch 2024-09-11T08:13:30.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:30 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.5", "format": "json"}]: dispatch 2024-09-11T08:13:30.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:30 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.5", "format": "json"}]': finished 2024-09-11T08:13:30.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:30 smithi122 ceph-mon[28992]: Reconfiguring daemon osd.5 on smithi135 2024-09-11T08:13:30.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:30 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/2965468748' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-11T08:13:30.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:30 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/3187313545' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-11T08:13:30.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:30 smithi122 ceph-mon[39726]: from='client.15177 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.5", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:13:30.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:30 smithi122 ceph-mon[39726]: Schedule rotate-key daemon osd.5 2024-09-11T08:13:30.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:30 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:13:30.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:30 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:13:30.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:30 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:13:30.986 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:30 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:13:30.986 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:30 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:13:30.986 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:30 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:13:30.986 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:30 smithi122 ceph-mon[39726]: Rotating authentication key for osd.5 2024-09-11T08:13:30.986 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:30 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.5", "format": "json"}]: dispatch 2024-09-11T08:13:30.986 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:30 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.5", "format": "json"}]: dispatch 2024-09-11T08:13:30.986 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:30 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.5", "format": "json"}]': finished 2024-09-11T08:13:30.986 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:30 smithi122 ceph-mon[39726]: Reconfiguring daemon osd.5 on smithi135 2024-09-11T08:13:31.977 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:31 smithi135 ceph-mon[32689]: pgmap v192: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:13:31.983 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:31 smithi122 ceph-mon[28992]: pgmap v192: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:13:31.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:31 smithi122 ceph-mon[39726]: pgmap v192: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:13:32.478 INFO:journalctl@ceph.iscsi.iscsi.a.smithi135.stdout:Sep 11 08:13:32 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[59799]: debug there is no tcmu-runner data available 2024-09-11T08:13:32.983 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:32 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:13:32.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:32 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:13:32.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:32 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:13:32.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:32 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:13:32.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:32 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:13:32.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:32 smithi122 ceph-mon[39726]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:13:32.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:32 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:13:32.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:32 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:13:32.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:32 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:13:32.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:32 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:13:32.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:32 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:13:32.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:32 smithi122 ceph-mon[28992]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:13:33.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:32 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:13:33.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:32 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:13:33.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:32 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:13:33.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:32 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:13:33.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:32 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:13:33.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:32 smithi135 ceph-mon[32689]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:13:33.733 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:13:33 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: ::ffff:172.21.15.135 - - [11/Sep/2024:08:13:33] "GET /metrics HTTP/1.1" 200 219537 "" "Prometheus/2.43.0" 2024-09-11T08:13:34.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:34 smithi135 ceph-mon[32689]: pgmap v193: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:13:34.733 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:34 smithi122 ceph-mon[28992]: pgmap v193: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:13:34.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:34 smithi122 ceph-mon[39726]: pgmap v193: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:13:35.305 INFO:teuthology.orchestra.run.smithi122.stderr:++ ceph auth get-key osd.5 2024-09-11T08:13:35.614 INFO:teuthology.orchestra.run.smithi122.stderr:+ NK=AQBBT+Fm7GU/IxAAolcMJmBKrSHvvukDJ475iw== 2024-09-11T08:13:35.615 INFO:teuthology.orchestra.run.smithi122.stderr:+ '[' AQBBT+Fm7GU/IxAAolcMJmBKrSHvvukDJ475iw== == AQBBT+Fm7GU/IxAAolcMJmBKrSHvvukDJ475iw== ']' 2024-09-11T08:13:35.615 INFO:teuthology.orchestra.run.smithi122.stderr:+ sleep 5 2024-09-11T08:13:36.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:36 smithi135 ceph-mon[32689]: pgmap v194: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:13:36.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:36 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/1117217757' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-11T08:13:36.733 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:36 smithi122 ceph-mon[39726]: pgmap v194: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:13:36.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:36 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/1117217757' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-11T08:13:36.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:36 smithi122 ceph-mon[28992]: pgmap v194: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:13:36.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:36 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/1117217757' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-11T08:13:38.228 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:13:37 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: ::ffff:172.21.15.135 - - [11/Sep/2024:08:13:37] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:13:38.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:38 smithi135 ceph-mon[32689]: pgmap v195: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:13:38.733 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:38 smithi122 ceph-mon[28992]: pgmap v195: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:13:38.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:38 smithi122 ceph-mon[39726]: pgmap v195: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:13:40.616 INFO:teuthology.orchestra.run.smithi122.stderr:++ ceph auth get-key osd.5 2024-09-11T08:13:40.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:40 smithi135 ceph-mon[32689]: pgmap v196: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:13:40.733 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:40 smithi122 ceph-mon[39726]: pgmap v196: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:13:40.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:40 smithi122 ceph-mon[28992]: pgmap v196: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:13:40.923 INFO:teuthology.orchestra.run.smithi122.stderr:+ NK=AQBBT+Fm7GU/IxAAolcMJmBKrSHvvukDJ475iw== 2024-09-11T08:13:40.923 INFO:teuthology.orchestra.run.smithi122.stderr:+ '[' AQBBT+Fm7GU/IxAAolcMJmBKrSHvvukDJ475iw== == AQBBT+Fm7GU/IxAAolcMJmBKrSHvvukDJ475iw== ']' 2024-09-11T08:13:40.923 INFO:teuthology.orchestra.run.smithi122.stderr:+ sleep 5 2024-09-11T08:13:41.727 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:41 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/2882201519' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-11T08:13:41.733 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:41 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/2882201519' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-11T08:13:41.733 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:41 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/2882201519' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-11T08:13:42.441 INFO:journalctl@ceph.iscsi.iscsi.a.smithi135.stdout:Sep 11 08:13:42 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[59799]: debug there is no tcmu-runner data available 2024-09-11T08:13:42.727 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:42 smithi135 ceph-mon[32689]: pgmap v197: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:13:42.733 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:42 smithi122 ceph-mon[39726]: pgmap v197: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:13:42.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:42 smithi122 ceph-mon[28992]: pgmap v197: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:13:43.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:43 smithi135 ceph-mon[32689]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:13:43.733 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:43 smithi122 ceph-mon[39726]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:13:43.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:43 smithi122 ceph-mon[28992]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:13:43.734 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:13:43 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: ::ffff:172.21.15.135 - - [11/Sep/2024:08:13:43] "GET /metrics HTTP/1.1" 200 219537 "" "Prometheus/2.43.0" 2024-09-11T08:13:44.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:44 smithi135 ceph-mon[32689]: pgmap v198: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:13:44.733 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:44 smithi122 ceph-mon[39726]: pgmap v198: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:13:44.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:44 smithi122 ceph-mon[28992]: pgmap v198: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:13:45.925 INFO:teuthology.orchestra.run.smithi122.stderr:++ ceph auth get-key osd.5 2024-09-11T08:13:46.236 INFO:teuthology.orchestra.run.smithi122.stderr:+ NK=AQBBT+Fm7GU/IxAAolcMJmBKrSHvvukDJ475iw== 2024-09-11T08:13:46.236 INFO:teuthology.orchestra.run.smithi122.stderr:+ '[' AQBBT+Fm7GU/IxAAolcMJmBKrSHvvukDJ475iw== == AQBBT+Fm7GU/IxAAolcMJmBKrSHvvukDJ475iw== ']' 2024-09-11T08:13:46.236 INFO:teuthology.orchestra.run.smithi122.stderr:+ sleep 5 2024-09-11T08:13:46.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:46 smithi135 ceph-mon[32689]: pgmap v199: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:13:46.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:46 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/1288506649' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-11T08:13:46.733 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:46 smithi122 ceph-mon[28992]: pgmap v199: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:13:46.733 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:46 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/1288506649' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-11T08:13:46.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:46 smithi122 ceph-mon[39726]: pgmap v199: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:13:46.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:46 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/1288506649' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-11T08:13:47.727 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:47 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:13:47.733 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:47 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:13:47.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:47 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:13:48.228 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:13:47 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: ::ffff:172.21.15.135 - - [11/Sep/2024:08:13:47] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:13:48.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:48 smithi135 ceph-mon[32689]: pgmap v200: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:13:48.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:48 smithi122 ceph-mon[28992]: pgmap v200: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:13:48.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:48 smithi122 ceph-mon[39726]: pgmap v200: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:13:50.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:50 smithi135 ceph-mon[32689]: pgmap v201: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:13:50.733 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:50 smithi122 ceph-mon[28992]: pgmap v201: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:13:50.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:50 smithi122 ceph-mon[39726]: pgmap v201: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:13:51.237 INFO:teuthology.orchestra.run.smithi122.stderr:++ ceph auth get-key osd.5 2024-09-11T08:13:51.555 INFO:teuthology.orchestra.run.smithi122.stderr:+ NK=AQBBT+Fm7GU/IxAAolcMJmBKrSHvvukDJ475iw== 2024-09-11T08:13:51.555 INFO:teuthology.orchestra.run.smithi122.stderr:+ '[' AQBBT+Fm7GU/IxAAolcMJmBKrSHvvukDJ475iw== == AQBBT+Fm7GU/IxAAolcMJmBKrSHvvukDJ475iw== ']' 2024-09-11T08:13:51.555 INFO:teuthology.orchestra.run.smithi122.stderr:+ sleep 5 2024-09-11T08:13:52.728 INFO:journalctl@ceph.iscsi.iscsi.a.smithi135.stdout:Sep 11 08:13:52 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[59799]: debug there is no tcmu-runner data available 2024-09-11T08:13:52.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:52 smithi135 ceph-mon[32689]: pgmap v202: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:13:52.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:52 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/4197309906' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-11T08:13:52.733 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:52 smithi122 ceph-mon[28992]: pgmap v202: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:13:52.733 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:52 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/4197309906' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-11T08:13:52.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:52 smithi122 ceph-mon[39726]: pgmap v202: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:13:52.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:52 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/4197309906' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-11T08:13:53.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:53 smithi135 ceph-mon[32689]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:13:53.733 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:53 smithi122 ceph-mon[28992]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:13:53.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:53 smithi122 ceph-mon[39726]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:13:53.734 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:13:53 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: ::ffff:172.21.15.135 - - [11/Sep/2024:08:13:53] "GET /metrics HTTP/1.1" 200 219532 "" "Prometheus/2.43.0" 2024-09-11T08:13:54.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:54 smithi135 ceph-mon[32689]: pgmap v203: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:13:54.983 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:54 smithi122 ceph-mon[28992]: pgmap v203: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:13:54.983 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:54 smithi122 ceph-mon[39726]: pgmap v203: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:13:55.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:55 smithi135 ceph-mon[32689]: pgmap v204: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:13:55.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:55 smithi122 ceph-mon[39726]: pgmap v204: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:13:55.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:55 smithi122 ceph-mon[28992]: pgmap v204: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:13:56.557 INFO:teuthology.orchestra.run.smithi122.stderr:++ ceph auth get-key osd.5 2024-09-11T08:13:56.869 INFO:teuthology.orchestra.run.smithi122.stdout:rotating key for osd.6 2024-09-11T08:13:56.869 INFO:teuthology.orchestra.run.smithi122.stderr:+ NK=AQAqUeFmK+Y0FhAA9wYNdNp8Ej22EscGBre+5A== 2024-09-11T08:13:56.870 INFO:teuthology.orchestra.run.smithi122.stderr:+ '[' AQBBT+Fm7GU/IxAAolcMJmBKrSHvvukDJ475iw== == AQAqUeFmK+Y0FhAA9wYNdNp8Ej22EscGBre+5A== ']' 2024-09-11T08:13:56.870 INFO:teuthology.orchestra.run.smithi122.stderr:+ for f in osd.0 osd.1 osd.2 osd.3 osd.4 osd.5 osd.6 osd.7 mgr.y mgr.x 2024-09-11T08:13:56.870 INFO:teuthology.orchestra.run.smithi122.stderr:+ echo 'rotating key for osd.6' 2024-09-11T08:13:56.870 INFO:teuthology.orchestra.run.smithi122.stderr:++ ceph auth get-key osd.6 2024-09-11T08:13:57.173 INFO:teuthology.orchestra.run.smithi122.stderr:+ K=AQBZT+FmK6dFNxAAGgRgcn/PQnojVDvo/1mKFQ== 2024-09-11T08:13:57.173 INFO:teuthology.orchestra.run.smithi122.stderr:+ NK=AQBZT+FmK6dFNxAAGgRgcn/PQnojVDvo/1mKFQ== 2024-09-11T08:13:57.173 INFO:teuthology.orchestra.run.smithi122.stderr:+ ceph orch daemon rotate-key osd.6 2024-09-11T08:13:57.174 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:56 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/1091232588' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-11T08:13:57.174 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:56 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/1091232588' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-11T08:13:57.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:56 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/1091232588' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-11T08:13:57.431 INFO:teuthology.orchestra.run.smithi122.stdout:Scheduled to rotate-key osd.6 on host 'smithi135' 2024-09-11T08:13:57.443 INFO:teuthology.orchestra.run.smithi122.stderr:+ '[' AQBZT+FmK6dFNxAAGgRgcn/PQnojVDvo/1mKFQ== == AQBZT+FmK6dFNxAAGgRgcn/PQnojVDvo/1mKFQ== ']' 2024-09-11T08:13:57.443 INFO:teuthology.orchestra.run.smithi122.stderr:+ sleep 5 2024-09-11T08:13:58.222 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:13:57 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: ::ffff:172.21.15.135 - - [11/Sep/2024:08:13:57] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:13:58.223 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:57 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/2157252765' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-11T08:13:58.223 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:57 smithi135 ceph-mon[32689]: pgmap v205: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:13:58.223 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:57 smithi135 ceph-mon[32689]: from='client.15231 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.6", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:13:58.223 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:57 smithi135 ceph-mon[32689]: Schedule rotate-key daemon osd.6 2024-09-11T08:13:58.223 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:57 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:13:58.223 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:57 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:13:58.223 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:57 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:13:58.223 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:57 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:13:58.223 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:57 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:13:58.223 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:57 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:13:58.224 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:57 smithi135 ceph-mon[32689]: Rotating authentication key for osd.6 2024-09-11T08:13:58.224 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:57 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.6", "format": "json"}]: dispatch 2024-09-11T08:13:58.224 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:57 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.6", "format": "json"}]: dispatch 2024-09-11T08:13:58.224 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:57 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.6", "format": "json"}]': finished 2024-09-11T08:13:58.224 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:57 smithi135 ceph-mon[32689]: Reconfiguring daemon osd.6 on smithi135 2024-09-11T08:13:58.234 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:57 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/2157252765' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-11T08:13:58.234 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:57 smithi122 ceph-mon[28992]: pgmap v205: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:13:58.234 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:57 smithi122 ceph-mon[28992]: from='client.15231 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.6", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:13:58.234 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:57 smithi122 ceph-mon[28992]: Schedule rotate-key daemon osd.6 2024-09-11T08:13:58.234 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:57 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:13:58.234 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:57 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:13:58.234 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:57 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:13:58.234 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:57 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:13:58.234 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:57 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:13:58.234 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:57 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:13:58.234 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:57 smithi122 ceph-mon[28992]: Rotating authentication key for osd.6 2024-09-11T08:13:58.234 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:57 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.6", "format": "json"}]: dispatch 2024-09-11T08:13:58.235 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:57 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.6", "format": "json"}]: dispatch 2024-09-11T08:13:58.235 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:57 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.6", "format": "json"}]': finished 2024-09-11T08:13:58.235 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:57 smithi122 ceph-mon[28992]: Reconfiguring daemon osd.6 on smithi135 2024-09-11T08:13:58.235 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:57 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/2157252765' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-11T08:13:58.235 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:57 smithi122 ceph-mon[39726]: pgmap v205: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:13:58.235 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:57 smithi122 ceph-mon[39726]: from='client.15231 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.6", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:13:58.235 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:57 smithi122 ceph-mon[39726]: Schedule rotate-key daemon osd.6 2024-09-11T08:13:58.235 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:57 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:13:58.235 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:57 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:13:58.235 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:57 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:13:58.235 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:57 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:13:58.235 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:57 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:13:58.235 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:57 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:13:58.236 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:57 smithi122 ceph-mon[39726]: Rotating authentication key for osd.6 2024-09-11T08:13:58.236 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:57 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.6", "format": "json"}]: dispatch 2024-09-11T08:13:58.236 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:57 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.6", "format": "json"}]: dispatch 2024-09-11T08:13:58.236 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:57 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.6", "format": "json"}]': finished 2024-09-11T08:13:58.236 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:57 smithi122 ceph-mon[39726]: Reconfiguring daemon osd.6 on smithi135 2024-09-11T08:14:00.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:59 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:14:00.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:59 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:14:00.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:59 smithi135 ceph-mon[32689]: pgmap v206: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:14:00.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:59 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:14:00.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:13:59 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:14:00.483 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:59 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:14:00.483 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:59 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:14:00.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:59 smithi122 ceph-mon[39726]: pgmap v206: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:14:00.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:59 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:14:00.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:13:59 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:14:00.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:59 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:14:00.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:59 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:14:00.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:59 smithi122 ceph-mon[28992]: pgmap v206: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:14:00.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:59 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:14:00.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:13:59 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:14:02.445 INFO:teuthology.orchestra.run.smithi122.stderr:++ ceph auth get-key osd.6 2024-09-11T08:14:02.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:02 smithi135 ceph-mon[32689]: pgmap v207: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:14:02.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:02 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:14:02.479 INFO:journalctl@ceph.iscsi.iscsi.a.smithi135.stdout:Sep 11 08:14:02 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[59799]: debug there is no tcmu-runner data available 2024-09-11T08:14:02.483 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:02 smithi122 ceph-mon[28992]: pgmap v207: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:14:02.483 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:02 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:14:02.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:02 smithi122 ceph-mon[39726]: pgmap v207: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:14:02.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:02 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:14:02.757 INFO:teuthology.orchestra.run.smithi122.stderr:+ NK=AQBZT+FmK6dFNxAAGgRgcn/PQnojVDvo/1mKFQ== 2024-09-11T08:14:02.757 INFO:teuthology.orchestra.run.smithi122.stderr:+ '[' AQBZT+FmK6dFNxAAGgRgcn/PQnojVDvo/1mKFQ== == AQBZT+FmK6dFNxAAGgRgcn/PQnojVDvo/1mKFQ== ']' 2024-09-11T08:14:02.757 INFO:teuthology.orchestra.run.smithi122.stderr:+ sleep 5 2024-09-11T08:14:03.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:03 smithi122 ceph-mon[39726]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:14:03.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:03 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/4154576673' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-11T08:14:03.484 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:03 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: ::ffff:172.21.15.135 - - [11/Sep/2024:08:14:03] "GET /metrics HTTP/1.1" 200 219527 "" "Prometheus/2.43.0" 2024-09-11T08:14:03.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:03 smithi122 ceph-mon[28992]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:14:03.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:03 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/4154576673' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-11T08:14:03.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:03 smithi135 ceph-mon[32689]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:14:03.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:03 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/4154576673' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-11T08:14:04.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:04 smithi122 ceph-mon[28992]: pgmap v208: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:14:04.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:04 smithi122 ceph-mon[39726]: pgmap v208: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:14:04.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:04 smithi135 ceph-mon[32689]: pgmap v208: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:14:06.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:06 smithi135 ceph-mon[32689]: pgmap v209: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:14:06.733 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:06 smithi122 ceph-mon[28992]: pgmap v209: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:14:06.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:06 smithi122 ceph-mon[39726]: pgmap v209: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:14:07.759 INFO:teuthology.orchestra.run.smithi122.stderr:++ ceph auth get-key osd.6 2024-09-11T08:14:08.067 INFO:teuthology.orchestra.run.smithi122.stderr:+ NK=AQBZT+FmK6dFNxAAGgRgcn/PQnojVDvo/1mKFQ== 2024-09-11T08:14:08.067 INFO:teuthology.orchestra.run.smithi122.stderr:+ '[' AQBZT+FmK6dFNxAAGgRgcn/PQnojVDvo/1mKFQ== == AQBZT+FmK6dFNxAAGgRgcn/PQnojVDvo/1mKFQ== ']' 2024-09-11T08:14:08.067 INFO:teuthology.orchestra.run.smithi122.stderr:+ sleep 5 2024-09-11T08:14:08.228 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:14:07 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: ::ffff:172.21.15.135 - - [11/Sep/2024:08:14:07] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:14:08.483 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:08 smithi122 ceph-mon[28992]: pgmap v210: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:14:08.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:08 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/943693715' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-11T08:14:08.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:08 smithi122 ceph-mon[39726]: pgmap v210: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:14:08.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:08 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/943693715' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-11T08:14:08.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:08 smithi135 ceph-mon[32689]: pgmap v210: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:14:08.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:08 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/943693715' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-11T08:14:10.727 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:10 smithi135 ceph-mon[32689]: pgmap v211: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:14:10.733 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:10 smithi122 ceph-mon[28992]: pgmap v211: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:14:10.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:10 smithi122 ceph-mon[39726]: pgmap v211: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:14:12.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:12 smithi135 ceph-mon[32689]: pgmap v212: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:14:12.728 INFO:journalctl@ceph.iscsi.iscsi.a.smithi135.stdout:Sep 11 08:14:12 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[59799]: debug there is no tcmu-runner data available 2024-09-11T08:14:12.733 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:12 smithi122 ceph-mon[39726]: pgmap v212: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:14:12.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:12 smithi122 ceph-mon[28992]: pgmap v212: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:14:13.069 INFO:teuthology.orchestra.run.smithi122.stderr:++ ceph auth get-key osd.6 2024-09-11T08:14:13.388 INFO:teuthology.orchestra.run.smithi122.stderr:+ NK=AQBZT+FmK6dFNxAAGgRgcn/PQnojVDvo/1mKFQ== 2024-09-11T08:14:13.388 INFO:teuthology.orchestra.run.smithi122.stderr:+ '[' AQBZT+FmK6dFNxAAGgRgcn/PQnojVDvo/1mKFQ== == AQBZT+FmK6dFNxAAGgRgcn/PQnojVDvo/1mKFQ== ']' 2024-09-11T08:14:13.388 INFO:teuthology.orchestra.run.smithi122.stderr:+ sleep 5 2024-09-11T08:14:13.389 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:13 smithi122 ceph-mon[28992]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:14:13.389 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:13 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: ::ffff:172.21.15.135 - - [11/Sep/2024:08:14:13] "GET /metrics HTTP/1.1" 200 219527 "" "Prometheus/2.43.0" 2024-09-11T08:14:13.389 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:13 smithi122 ceph-mon[39726]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:14:13.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:13 smithi135 ceph-mon[32689]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:14:14.727 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:14 smithi135 ceph-mon[32689]: pgmap v213: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:14:14.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:14 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/422892571' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-11T08:14:14.733 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:14 smithi122 ceph-mon[28992]: pgmap v213: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:14:14.733 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:14 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/422892571' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-11T08:14:14.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:14 smithi122 ceph-mon[39726]: pgmap v213: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:14:14.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:14 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/422892571' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-11T08:14:16.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:16 smithi135 ceph-mon[32689]: pgmap v214: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:14:16.733 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:16 smithi122 ceph-mon[28992]: pgmap v214: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:14:16.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:16 smithi122 ceph-mon[39726]: pgmap v214: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:14:17.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:17 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:14:17.733 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:17 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:14:17.733 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:17 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:14:18.228 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:14:17 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: ::ffff:172.21.15.135 - - [11/Sep/2024:08:14:17] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:14:18.390 INFO:teuthology.orchestra.run.smithi122.stderr:++ ceph auth get-key osd.6 2024-09-11T08:14:18.703 INFO:teuthology.orchestra.run.smithi122.stdout:rotating key for osd.7 2024-09-11T08:14:18.704 INFO:teuthology.orchestra.run.smithi122.stderr:+ NK=AQBFUeFmyyDKHhAAF7RtSTbOgPt572omzM8laQ== 2024-09-11T08:14:18.704 INFO:teuthology.orchestra.run.smithi122.stderr:+ '[' AQBZT+FmK6dFNxAAGgRgcn/PQnojVDvo/1mKFQ== == AQBFUeFmyyDKHhAAF7RtSTbOgPt572omzM8laQ== ']' 2024-09-11T08:14:18.704 INFO:teuthology.orchestra.run.smithi122.stderr:+ for f in osd.0 osd.1 osd.2 osd.3 osd.4 osd.5 osd.6 osd.7 mgr.y mgr.x 2024-09-11T08:14:18.704 INFO:teuthology.orchestra.run.smithi122.stderr:+ echo 'rotating key for osd.7' 2024-09-11T08:14:18.704 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:18 smithi122 ceph-mon[39726]: pgmap v215: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:14:18.705 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:18 smithi122 ceph-mon[28992]: pgmap v215: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:14:18.705 INFO:teuthology.orchestra.run.smithi122.stderr:++ ceph auth get-key osd.7 2024-09-11T08:14:18.727 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:18 smithi135 ceph-mon[32689]: pgmap v215: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:14:19.007 INFO:teuthology.orchestra.run.smithi122.stderr:+ K=AQBxT+Fm6xI+KxAA40hwaz070A2wU9p94SKPBA== 2024-09-11T08:14:19.007 INFO:teuthology.orchestra.run.smithi122.stderr:+ NK=AQBxT+Fm6xI+KxAA40hwaz070A2wU9p94SKPBA== 2024-09-11T08:14:19.007 INFO:teuthology.orchestra.run.smithi122.stderr:+ ceph orch daemon rotate-key osd.7 2024-09-11T08:14:19.268 INFO:teuthology.orchestra.run.smithi122.stdout:Scheduled to rotate-key osd.7 on host 'smithi135' 2024-09-11T08:14:19.279 INFO:teuthology.orchestra.run.smithi122.stderr:+ '[' AQBxT+Fm6xI+KxAA40hwaz070A2wU9p94SKPBA== == AQBxT+Fm6xI+KxAA40hwaz070A2wU9p94SKPBA== ']' 2024-09-11T08:14:19.279 INFO:teuthology.orchestra.run.smithi122.stderr:+ sleep 5 2024-09-11T08:14:19.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:19 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/3187486305' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-11T08:14:19.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:19 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/3439181124' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-11T08:14:19.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:19 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:14:19.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:19 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:14:19.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:19 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:14:19.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:19 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:14:19.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:19 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:14:19.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:19 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:14:19.983 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:19 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/3187486305' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-11T08:14:19.983 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:19 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/3439181124' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-11T08:14:19.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:19 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:14:19.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:19 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:14:19.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:19 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:14:19.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:19 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:14:19.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:19 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:14:19.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:19 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:14:19.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:19 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/3187486305' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-11T08:14:19.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:19 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/3439181124' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-11T08:14:19.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:19 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:14:19.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:19 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:14:19.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:19 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:14:19.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:19 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:14:19.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:19 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:14:19.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:19 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:14:20.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:20 smithi135 ceph-mon[32689]: pgmap v216: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:14:20.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:20 smithi135 ceph-mon[32689]: from='client.25181 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.7", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:14:20.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:20 smithi135 ceph-mon[32689]: Schedule rotate-key daemon osd.7 2024-09-11T08:14:20.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:20 smithi135 ceph-mon[32689]: Rotating authentication key for osd.7 2024-09-11T08:14:20.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:20 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.7", "format": "json"}]: dispatch 2024-09-11T08:14:20.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:20 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.7", "format": "json"}]: dispatch 2024-09-11T08:14:20.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:20 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.7", "format": "json"}]': finished 2024-09-11T08:14:20.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:20 smithi135 ceph-mon[32689]: Reconfiguring daemon osd.7 on smithi135 2024-09-11T08:14:20.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:20 smithi122 ceph-mon[28992]: pgmap v216: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:14:20.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:20 smithi122 ceph-mon[28992]: from='client.25181 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.7", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:14:20.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:20 smithi122 ceph-mon[28992]: Schedule rotate-key daemon osd.7 2024-09-11T08:14:20.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:20 smithi122 ceph-mon[28992]: Rotating authentication key for osd.7 2024-09-11T08:14:20.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:20 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.7", "format": "json"}]: dispatch 2024-09-11T08:14:20.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:20 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.7", "format": "json"}]: dispatch 2024-09-11T08:14:20.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:20 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.7", "format": "json"}]': finished 2024-09-11T08:14:20.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:20 smithi122 ceph-mon[28992]: Reconfiguring daemon osd.7 on smithi135 2024-09-11T08:14:20.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:20 smithi122 ceph-mon[39726]: pgmap v216: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:14:20.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:20 smithi122 ceph-mon[39726]: from='client.25181 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.7", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:14:20.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:20 smithi122 ceph-mon[39726]: Schedule rotate-key daemon osd.7 2024-09-11T08:14:20.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:20 smithi122 ceph-mon[39726]: Rotating authentication key for osd.7 2024-09-11T08:14:20.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:20 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.7", "format": "json"}]: dispatch 2024-09-11T08:14:20.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:20 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.7", "format": "json"}]: dispatch 2024-09-11T08:14:20.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:20 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.7", "format": "json"}]': finished 2024-09-11T08:14:20.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:20 smithi122 ceph-mon[39726]: Reconfiguring daemon osd.7 on smithi135 2024-09-11T08:14:22.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:22 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:14:22.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:22 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:14:22.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:22 smithi135 ceph-mon[32689]: pgmap v217: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:14:22.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:22 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:14:22.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:22 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:14:22.479 INFO:journalctl@ceph.iscsi.iscsi.a.smithi135.stdout:Sep 11 08:14:22 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[59799]: debug there is no tcmu-runner data available 2024-09-11T08:14:22.483 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:22 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:14:22.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:22 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:14:22.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:22 smithi122 ceph-mon[39726]: pgmap v217: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:14:22.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:22 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:14:22.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:22 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:14:22.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:22 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:14:22.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:22 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:14:22.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:22 smithi122 ceph-mon[28992]: pgmap v217: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:14:22.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:22 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:14:22.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:22 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:14:23.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:23 smithi135 ceph-mon[32689]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:14:23.483 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:23 smithi122 ceph-mon[28992]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:14:23.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:23 smithi122 ceph-mon[39726]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:14:23.484 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:23 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: ::ffff:172.21.15.135 - - [11/Sep/2024:08:14:23] "GET /metrics HTTP/1.1" 200 219535 "" "Prometheus/2.43.0" 2024-09-11T08:14:24.281 INFO:teuthology.orchestra.run.smithi122.stderr:++ ceph auth get-key osd.7 2024-09-11T08:14:24.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:24 smithi135 ceph-mon[32689]: pgmap v218: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:14:24.483 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:24 smithi122 ceph-mon[28992]: pgmap v218: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:14:24.483 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:24 smithi122 ceph-mon[39726]: pgmap v218: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:14:24.591 INFO:teuthology.orchestra.run.smithi122.stderr:+ NK=AQBxT+Fm6xI+KxAA40hwaz070A2wU9p94SKPBA== 2024-09-11T08:14:24.592 INFO:teuthology.orchestra.run.smithi122.stderr:+ '[' AQBxT+Fm6xI+KxAA40hwaz070A2wU9p94SKPBA== == AQBxT+Fm6xI+KxAA40hwaz070A2wU9p94SKPBA== ']' 2024-09-11T08:14:24.592 INFO:teuthology.orchestra.run.smithi122.stderr:+ sleep 5 2024-09-11T08:14:25.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:25 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/3630108719' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-11T08:14:25.483 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:25 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/3630108719' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-11T08:14:25.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:25 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/3630108719' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-11T08:14:26.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:26 smithi135 ceph-mon[32689]: pgmap v219: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:14:26.483 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:26 smithi122 ceph-mon[28992]: pgmap v219: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:14:26.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:26 smithi122 ceph-mon[39726]: pgmap v219: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:14:28.227 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:14:27 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: ::ffff:172.21.15.135 - - [11/Sep/2024:08:14:27] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:14:28.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:28 smithi122 ceph-mon[28992]: pgmap v220: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:14:28.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:28 smithi122 ceph-mon[39726]: pgmap v220: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:14:28.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:28 smithi135 ceph-mon[32689]: pgmap v220: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:14:29.593 INFO:teuthology.orchestra.run.smithi122.stderr:++ ceph auth get-key osd.7 2024-09-11T08:14:29.907 INFO:teuthology.orchestra.run.smithi122.stderr:+ NK=AQBxT+Fm6xI+KxAA40hwaz070A2wU9p94SKPBA== 2024-09-11T08:14:29.907 INFO:teuthology.orchestra.run.smithi122.stderr:+ '[' AQBxT+Fm6xI+KxAA40hwaz070A2wU9p94SKPBA== == AQBxT+Fm6xI+KxAA40hwaz070A2wU9p94SKPBA== ']' 2024-09-11T08:14:29.907 INFO:teuthology.orchestra.run.smithi122.stderr:+ sleep 5 2024-09-11T08:14:30.530 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:30 smithi135 ceph-mon[32689]: pgmap v221: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:14:30.530 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:30 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/740001870' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-11T08:14:30.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:30 smithi122 ceph-mon[28992]: pgmap v221: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:14:30.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:30 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/740001870' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-11T08:14:30.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:30 smithi122 ceph-mon[39726]: pgmap v221: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:14:30.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:30 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/740001870' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-11T08:14:32.727 INFO:journalctl@ceph.iscsi.iscsi.a.smithi135.stdout:Sep 11 08:14:32 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[59799]: debug there is no tcmu-runner data available 2024-09-11T08:14:32.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:32 smithi135 ceph-mon[32689]: pgmap v222: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:14:32.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:32 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:14:32.733 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:32 smithi122 ceph-mon[28992]: pgmap v222: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:14:32.733 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:32 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:14:32.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:32 smithi122 ceph-mon[39726]: pgmap v222: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:14:32.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:32 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:14:33.727 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:33 smithi135 ceph-mon[32689]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:14:33.733 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:33 smithi122 ceph-mon[28992]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:14:33.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:33 smithi122 ceph-mon[39726]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:14:33.734 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:33 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: ::ffff:172.21.15.135 - - [11/Sep/2024:08:14:33] "GET /metrics HTTP/1.1" 200 219540 "" "Prometheus/2.43.0" 2024-09-11T08:14:34.727 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:34 smithi135 ceph-mon[32689]: pgmap v223: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:14:34.733 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:34 smithi122 ceph-mon[28992]: pgmap v223: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:14:34.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:34 smithi122 ceph-mon[39726]: pgmap v223: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:14:34.909 INFO:teuthology.orchestra.run.smithi122.stderr:++ ceph auth get-key osd.7 2024-09-11T08:14:35.221 INFO:teuthology.orchestra.run.smithi122.stderr:+ NK=AQBxT+Fm6xI+KxAA40hwaz070A2wU9p94SKPBA== 2024-09-11T08:14:35.221 INFO:teuthology.orchestra.run.smithi122.stderr:+ '[' AQBxT+Fm6xI+KxAA40hwaz070A2wU9p94SKPBA== == AQBxT+Fm6xI+KxAA40hwaz070A2wU9p94SKPBA== ']' 2024-09-11T08:14:35.221 INFO:teuthology.orchestra.run.smithi122.stderr:+ sleep 5 2024-09-11T08:14:35.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:35 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/145895934' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-11T08:14:35.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:35 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/145895934' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-11T08:14:35.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:35 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/145895934' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-11T08:14:36.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:36 smithi135 ceph-mon[32689]: pgmap v224: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:14:36.733 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:36 smithi122 ceph-mon[28992]: pgmap v224: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:14:36.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:36 smithi122 ceph-mon[39726]: pgmap v224: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:14:38.228 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:14:37 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: ::ffff:172.21.15.135 - - [11/Sep/2024:08:14:37] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:14:38.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:38 smithi135 ceph-mon[32689]: pgmap v225: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:14:38.733 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:38 smithi122 ceph-mon[39726]: pgmap v225: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:14:38.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:38 smithi122 ceph-mon[28992]: pgmap v225: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:14:40.223 INFO:teuthology.orchestra.run.smithi122.stderr:++ ceph auth get-key osd.7 2024-09-11T08:14:40.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:40 smithi122 ceph-mon[28992]: pgmap v226: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:14:40.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:40 smithi122 ceph-mon[39726]: pgmap v226: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:14:40.534 INFO:teuthology.orchestra.run.smithi122.stderr:+ NK=AQBxT+Fm6xI+KxAA40hwaz070A2wU9p94SKPBA== 2024-09-11T08:14:40.535 INFO:teuthology.orchestra.run.smithi122.stderr:+ '[' AQBxT+Fm6xI+KxAA40hwaz070A2wU9p94SKPBA== == AQBxT+Fm6xI+KxAA40hwaz070A2wU9p94SKPBA== ']' 2024-09-11T08:14:40.535 INFO:teuthology.orchestra.run.smithi122.stderr:+ sleep 5 2024-09-11T08:14:40.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:40 smithi135 ceph-mon[32689]: pgmap v226: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:14:41.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:41 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/4068957637' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-11T08:14:41.733 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:41 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/4068957637' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-11T08:14:41.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:41 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/4068957637' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-11T08:14:42.727 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:42 smithi135 ceph-mon[32689]: pgmap v227: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:14:42.728 INFO:journalctl@ceph.iscsi.iscsi.a.smithi135.stdout:Sep 11 08:14:42 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[59799]: debug there is no tcmu-runner data available 2024-09-11T08:14:42.733 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:42 smithi122 ceph-mon[28992]: pgmap v227: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:14:42.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:42 smithi122 ceph-mon[39726]: pgmap v227: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:14:43.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:43 smithi135 ceph-mon[32689]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:14:43.733 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:43 smithi122 ceph-mon[39726]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:14:43.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:43 smithi122 ceph-mon[28992]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:14:43.734 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:43 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: ::ffff:172.21.15.135 - - [11/Sep/2024:08:14:43] "GET /metrics HTTP/1.1" 200 219540 "" "Prometheus/2.43.0" 2024-09-11T08:14:44.977 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:44 smithi135 ceph-mon[32689]: pgmap v228: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:14:44.983 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:44 smithi122 ceph-mon[39726]: pgmap v228: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:14:44.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:44 smithi122 ceph-mon[28992]: pgmap v228: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:14:45.536 INFO:teuthology.orchestra.run.smithi122.stderr:++ ceph auth get-key osd.7 2024-09-11T08:14:45.854 INFO:teuthology.orchestra.run.smithi122.stdout:rotating key for mgr.y 2024-09-11T08:14:45.854 INFO:teuthology.orchestra.run.smithi122.stderr:+ NK=AQBbUeFm7WbnKhAAnbmboewVmllWfWCFvbMTsg== 2024-09-11T08:14:45.854 INFO:teuthology.orchestra.run.smithi122.stderr:+ '[' AQBxT+Fm6xI+KxAA40hwaz070A2wU9p94SKPBA== == AQBbUeFm7WbnKhAAnbmboewVmllWfWCFvbMTsg== ']' 2024-09-11T08:14:45.854 INFO:teuthology.orchestra.run.smithi122.stderr:+ for f in osd.0 osd.1 osd.2 osd.3 osd.4 osd.5 osd.6 osd.7 mgr.y mgr.x 2024-09-11T08:14:45.855 INFO:teuthology.orchestra.run.smithi122.stderr:+ echo 'rotating key for mgr.y' 2024-09-11T08:14:45.855 INFO:teuthology.orchestra.run.smithi122.stderr:++ ceph auth get-key mgr.y 2024-09-11T08:14:46.154 INFO:teuthology.orchestra.run.smithi122.stderr:+ K=AQBZTuFml29WBxAAvLofKoNFoe1uzS337O1tEA== 2024-09-11T08:14:46.154 INFO:teuthology.orchestra.run.smithi122.stderr:+ NK=AQBZTuFml29WBxAAvLofKoNFoe1uzS337O1tEA== 2024-09-11T08:14:46.154 INFO:teuthology.orchestra.run.smithi122.stderr:+ ceph orch daemon rotate-key mgr.y 2024-09-11T08:14:46.407 INFO:teuthology.orchestra.run.smithi122.stdout:Scheduled to rotate-key mgr.y on host 'smithi122' 2024-09-11T08:14:46.419 INFO:teuthology.orchestra.run.smithi122.stderr:+ '[' AQBZTuFml29WBxAAvLofKoNFoe1uzS337O1tEA== == AQBZTuFml29WBxAAvLofKoNFoe1uzS337O1tEA== ']' 2024-09-11T08:14:46.419 INFO:teuthology.orchestra.run.smithi122.stderr:+ sleep 5 2024-09-11T08:14:46.977 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:46 smithi122 ceph-mon[39726]: pgmap v229: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:14:46.977 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:46 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/1556213646' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-11T08:14:46.977 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:46 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/1140159902' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.y"}]: dispatch 2024-09-11T08:14:46.977 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:46 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:14:46.977 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:46 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:14:46.977 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:46 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:14:46.977 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:46 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:14:46.977 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:46 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:14:46.977 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:46 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:14:46.978 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:46 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]: dispatch 2024-09-11T08:14:46.978 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:46 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]: dispatch 2024-09-11T08:14:46.978 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:46 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]': finished 2024-09-11T08:14:46.978 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:46 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T08:14:46.978 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:46 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T08:14:46.978 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:46 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T08:14:46.978 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:46 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:14:46.978 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:46 smithi122 ceph-mon[28992]: pgmap v229: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:14:46.978 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:46 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/1556213646' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-11T08:14:46.978 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:46 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/1140159902' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.y"}]: dispatch 2024-09-11T08:14:46.979 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:46 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:14:46.979 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:46 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:14:46.979 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:46 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:14:46.979 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:46 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:14:46.979 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:46 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:14:46.979 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:46 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:14:46.979 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:46 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]: dispatch 2024-09-11T08:14:46.979 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:46 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]: dispatch 2024-09-11T08:14:46.979 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:46 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]': finished 2024-09-11T08:14:46.979 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:46 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T08:14:46.979 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:46 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T08:14:46.979 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:46 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T08:14:46.979 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:46 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:14:46.980 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:46 smithi135 ceph-mon[32689]: pgmap v229: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:14:46.980 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:46 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/1556213646' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-11T08:14:46.980 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:46 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/1140159902' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.y"}]: dispatch 2024-09-11T08:14:46.980 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:46 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:14:46.980 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:46 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:14:46.980 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:46 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:14:46.980 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:46 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:14:46.980 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:46 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:14:46.980 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:46 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:14:46.980 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:46 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]: dispatch 2024-09-11T08:14:46.980 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:46 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]: dispatch 2024-09-11T08:14:46.981 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:46 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]': finished 2024-09-11T08:14:46.981 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:46 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T08:14:46.981 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:46 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T08:14:46.981 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:46 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T08:14:46.981 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:46 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:14:47.935 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:47 smithi135 ceph-mon[32689]: from='client.15333 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "mgr.y", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:14:47.936 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:47 smithi135 ceph-mon[32689]: Schedule rotate-key daemon mgr.y 2024-09-11T08:14:47.936 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:47 smithi135 ceph-mon[32689]: Rotating authentication key for mgr.y 2024-09-11T08:14:47.936 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:47 smithi135 ceph-mon[32689]: Reconfiguring daemon mgr.y on smithi122 2024-09-11T08:14:47.936 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:47 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:14:47.936 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:47 smithi135 ceph-mon[32689]: pgmap v230: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:14:47.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:47 smithi122 ceph-mon[28992]: from='client.15333 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "mgr.y", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:14:47.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:47 smithi122 ceph-mon[28992]: Schedule rotate-key daemon mgr.y 2024-09-11T08:14:47.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:47 smithi122 ceph-mon[28992]: Rotating authentication key for mgr.y 2024-09-11T08:14:47.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:47 smithi122 ceph-mon[28992]: Reconfiguring daemon mgr.y on smithi122 2024-09-11T08:14:47.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:47 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:14:47.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:47 smithi122 ceph-mon[28992]: pgmap v230: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:14:47.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:47 smithi122 ceph-mon[39726]: from='client.15333 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "mgr.y", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:14:47.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:47 smithi122 ceph-mon[39726]: Schedule rotate-key daemon mgr.y 2024-09-11T08:14:47.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:47 smithi122 ceph-mon[39726]: Rotating authentication key for mgr.y 2024-09-11T08:14:47.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:47 smithi122 ceph-mon[39726]: Reconfiguring daemon mgr.y on smithi122 2024-09-11T08:14:47.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:47 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:14:47.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:47 smithi122 ceph-mon[39726]: pgmap v230: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:14:48.228 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:14:47 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: ::ffff:172.21.15.135 - - [11/Sep/2024:08:14:47] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:14:48.228 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:14:48 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: [11/Sep/2024:08:14:48] ENGINE Bus STOPPING 2024-09-11T08:14:48.338 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:14:48.070+0000 7fb1edf1e640 -1 mgr.server reply reply (13) Permission denied access denied: does your client key have mgr caps? See http://docs.ceph.com/en/latest/mgr/administrator/#client-authentication 2024-09-11T08:14:48.338 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:14:48.072+0000 7fb1dfe82640 -1 log_channel(cephadm) log [ERR] : Non-zero return from ['ceph', '-k', '/var/lib/ceph/mgr/ceph-y/keyring', '-n', 'mgr.y', 'tell', 'mgr.y', 'rotate-key', '-i', '-']: 2024-09-11T08:14:48.065+0000 7f40bf7fe640 1 -- 172.21.15.122:0/3046554155 <== mon.0 v2:172.21.15.122:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x7f40b0003780 con 0x7f40c01070c0 2024-09-11T08:14:48.338 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:14:48.065+0000 7f40bf7fe640 0 cephx client: could not set rotating key: decode_decrypt failed. error:bad magic in decode_decrypt, 5995656990721271821 != 18374858748799134293 2024-09-11T08:14:48.338 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:14:48.066+0000 7f40c6feb640 1 -- 172.21.15.122:0/3046554155 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f40c01070c0 msgr2=0x7f40c01094a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:14:48.338 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:14:48.066+0000 7f40c6feb640 1 --2- 172.21.15.122:0/3046554155 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f40c01070c0 0x7f40c01094a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f40ac004950 tx=0x7f40ac009da0 comp rx=0 tx=0).stop 2024-09-11T08:14:48.338 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:14:48.066+0000 7f40c6feb640 1 -- 172.21.15.122:0/3046554155 shutdown_connections 2024-09-11T08:14:48.339 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:14:48.066+0000 7f40c6feb640 1 --2- 172.21.15.122:0/3046554155 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f40c0109b30 0x7f40c010bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:14:48.339 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:14:48.066+0000 7f40c6feb640 1 --2- 172.21.15.122:0/3046554155 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f40c01070c0 0x7f40c01094a0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:14:48.339 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:14:48.066+0000 7f40c6feb640 1 --2- 172.21.15.122:0/3046554155 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f40c0104710 0x7f40c0106af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:14:48.339 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:14:48.066+0000 7f40c6feb640 1 -- 172.21.15.122:0/3046554155 >> 172.21.15.122:0/3046554155 conn(0x7f40c006a8a0 msgr2=0x7f40c006ccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:14:48.339 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:14:48.066+0000 7f40c6feb640 1 -- 172.21.15.122:0/3046554155 shutdown_connections 2024-09-11T08:14:48.339 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:14:48.066+0000 7f40c6feb640 1 -- 172.21.15.122:0/3046554155 wait complete. 2024-09-11T08:14:48.339 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:14:48.066+0000 7f40c6feb640 1 Processor -- start 2024-09-11T08:14:48.339 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:14:48.066+0000 7f40c6feb640 1 -- start start 2024-09-11T08:14:48.339 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:14:48.066+0000 7f40c6feb640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f40c0104710 0x7f40c00669d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:14:48.339 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:14:48.066+0000 7f40c6feb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f40c01070c0 0x7f40c011a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:14:48.339 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:14:48.066+0000 7f40c6feb640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f40c0109b30 0x7f40c011c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:14:48.340 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:14:48.066+0000 7f40c6feb640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c011d0b0 con 0x7f40c01070c0 2024-09-11T08:14:48.340 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:14:48.066+0000 7f40c6feb640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c011d220 con 0x7f40c0109b30 2024-09-11T08:14:48.340 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:14:48.066+0000 7f40c6feb640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40c011d3e0 con 0x7f40c0104710 2024-09-11T08:14:48.340 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:14:48.067+0000 7f40bffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f40c01070c0 0x7f40c011a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:14:48.340 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:14:48.067+0000 7f40c4d87640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f40c0104710 0x7f40c00669d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:14:48.340 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:14:48.067+0000 7f40bffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f40c01070c0 0x7f40c011a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33052/0 (socket says 172.21.15.122:33052) 2024-09-11T08:14:48.340 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:14:48.067+0000 7f40bffff640 1 -- 172.21.15.122:0/777778971 learned_addr learned my addr 172.21.15.122:0/777778971 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-11T08:14:48.340 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:14:48.067+0000 7f40c5588640 1 --2- 172.21.15.122:0/777778971 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f40c0109b30 0x7f40c011c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:14:48.340 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:14:48.067+0000 7f40bffff640 1 -- 172.21.15.122:0/777778971 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f40c0104710 msgr2=0x7f40c00669d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:14:48.340 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:14:48.067+0000 7f40bffff640 1 --2- 172.21.15.122:0/777778971 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f40c0104710 0x7f40c00669d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:14:48.340 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:14:48.067+0000 7f40bffff640 1 -- 172.21.15.122:0/777778971 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f40c0109b30 msgr2=0x7f40c011c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:14:48.341 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:14:48.067+0000 7f40bffff640 1 --2- 172.21.15.122:0/777778971 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f40c0109b30 0x7f40c011c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:14:48.341 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:14:48.067+0000 7f40bffff640 1 -- 172.21.15.122:0/777778971 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40b80098b0 con 0x7f40c01070c0 2024-09-11T08:14:48.341 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:14:48.067+0000 7f40c4d87640 1 --2- 172.21.15.122:0/777778971 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f40c0104710 0x7f40c00669d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T08:14:48.341 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:14:48.067+0000 7f40bffff640 1 --2- 172.21.15.122:0/777778971 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f40c01070c0 0x7f40c011a370 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f40ac02a4a0 tx=0x7f40ac009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:14:48.341 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:14:48.067+0000 7f40bd7fa640 1 -- 172.21.15.122:0/777778971 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40ac02a960 con 0x7f40c01070c0 2024-09-11T08:14:48.341 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:14:48.067+0000 7f40bd7fa640 1 -- 172.21.15.122:0/777778971 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- auth(proto 2 2 bytes epoch 0) v1 -- 0x7f40b00032d0 con 0x7f40c01070c0 2024-09-11T08:14:48.341 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:14:48.067+0000 7f40bd7fa640 1 -- 172.21.15.122:0/777778971 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(36 keys) v1 ==== 1701+0+0 (secure 0 0 0) 0x7f40ac0097f0 con 0x7f40c01070c0 2024-09-11T08:14:48.341 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:14:48.067+0000 7f40c6feb640 1 -- 172.21.15.122:0/777778971 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40ac005040 con 0x7f40c01070c0 2024-09-11T08:14:48.341 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:14:48.067+0000 7f40bd7fa640 1 -- 172.21.15.122:0/777778971 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40ac033420 con 0x7f40c01070c0 2024-09-11T08:14:48.341 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:14:48.067+0000 7f40c6feb640 1 -- 172.21.15.122:0/777778971 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40c011d940 con 0x7f40c01070c0 2024-09-11T08:14:48.341 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:14:48.067+0000 7f40bd7fa640 1 -- 172.21.15.122:0/777778971 <== mon.0 v2:172.21.15.122:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x7f40c011d940 con 0x7f40c01070c0 2024-09-11T08:14:48.341 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:14:48.067+0000 7f40bd7fa640 0 cephx client: could not set rotating key: decode_decrypt failed. error:bad magic in decode_decrypt, 5995656990721271821 != 18374858748799134293 2024-09-11T08:14:48.342 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:14:48.069+0000 7f40bd7fa640 1 -- 172.21.15.122:0/777778971 <== mon.0 v2:172.21.15.122:3300/0 5 ==== mgrmap(e 21) v1 ==== 90839+0+0 (secure 0 0 0) 0x7f40ac03a070 con 0x7f40c01070c0 2024-09-11T08:14:48.342 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:14:48.069+0000 7f40bd7fa640 1 --2- 172.21.15.122:0/777778971 >> [v2:172.21.15.122:6800/153625737,v1:172.21.15.122:6801/153625737] conn(0x7f40b006f1b0 0x7f40b0071660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:14:48.342 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:14:48.069+0000 7f40bd7fa640 1 -- 172.21.15.122:0/777778971 --> [v2:172.21.15.122:6800/153625737,v1:172.21.15.122:6801/153625737] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f40b0071d70 con 0x7f40b006f1b0 2024-09-11T08:14:48.342 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:14:48.069+0000 7f40bd7fa640 1 -- 172.21.15.122:0/777778971 <== mon.0 v2:172.21.15.122:3300/0 6 ==== osd_map(63..63 src has 1..63) v4 ==== 7533+0+0 (secure 0 0 0) 0x7f40ac0af9e0 con 0x7f40c01070c0 2024-09-11T08:14:48.342 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:14:48.069+0000 7f40c4d87640 1 --2- 172.21.15.122:0/777778971 >> [v2:172.21.15.122:6800/153625737,v1:172.21.15.122:6801/153625737] conn(0x7f40b006f1b0 0x7f40b0071660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:14:48.342 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:14:48.070+0000 7f40c4d87640 1 --2- 172.21.15.122:0/777778971 >> [v2:172.21.15.122:6800/153625737,v1:172.21.15.122:6801/153625737] conn(0x7f40b006f1b0 0x7f40b0071660 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f40c0066760 tx=0x7f40b4009120 comp rx=0 tx=0).ready entity=mgr.24440 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:14:48.342 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:14:48.070+0000 7f40bd7fa640 1 -- 172.21.15.122:0/777778971 <== mgr.24440 v2:172.21.15.122:6800/153625737 1 ==== command_reply(tid 0: -13 access denied: does your client key have mgr caps? See http://docs.ceph.com/en/latest/mgr/administrator/#client-authentication) v1 ==== 134+0+0 (secure 0 0 0) 0x7f40b0071d70 con 0x7f40b006f1b0 2024-09-11T08:14:48.342 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: Error EACCES: problem getting command descriptions from mgr.y 2024-09-11T08:14:48.343 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:14:48.343 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:14:48.083+0000 7fb20492d640 -1 mgr handle_mgr_map I was active but no longer am 2024-09-11T08:14:48.343 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: ignoring --setuser ceph since I am not root 2024-09-11T08:14:48.343 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: ignoring --setgroup ceph since I am not root 2024-09-11T08:14:48.343 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-mgr[29221]: -- 172.21.15.122:0/3033754702 <== mon.2 v2:172.21.15.122:3301/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x559738436340 con 0x559738442800 2024-09-11T08:14:48.343 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-mgr[29221]: cephx client: could not set rotating key: decode_decrypt failed. error:bad magic in decode_decrypt, 5995656990721271821 != 18374858748799134293 2024-09-11T08:14:48.343 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:14:48.217+0000 7f6c5fc34ec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T08:14:48.343 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:14:48.271+0000 7f6c5fc34ec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T08:14:48.343 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:14:48.336+0000 7f6c5fc34ec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T08:14:48.600 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:14:48.460+0000 7f6c5fc34ec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T08:14:48.696 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:14:48 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: [11/Sep/2024:08:14:48] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-11T08:14:48.696 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:14:48 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: [11/Sep/2024:08:14:48] ENGINE Bus STOPPED 2024-09-11T08:14:48.696 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:14:48 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: [11/Sep/2024:08:14:48] ENGINE Bus STARTING 2024-09-11T08:14:48.696 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:14:48 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: [11/Sep/2024:08:14:48] ENGINE Serving on http://:::9283 2024-09-11T08:14:48.696 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:14:48 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: [11/Sep/2024:08:14:48] ENGINE Bus STARTED 2024-09-11T08:14:48.866 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:14:48.599+0000 7f6c5fc34ec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T08:14:49.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:48 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:14:49.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:48 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:14:49.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:48 smithi135 ceph-mon[32689]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "mgr fail", "who": "y"}]: dispatch 2024-09-11T08:14:49.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:48 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' cmd=[{"prefix": "mgr fail", "who": "y"}]: dispatch 2024-09-11T08:14:49.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:48 smithi135 ceph-mon[32689]: Activating manager daemon x 2024-09-11T08:14:49.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:48 smithi135 ceph-mon[32689]: osdmap e64: 8 total, 8 up, 8 in 2024-09-11T08:14:49.229 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:48 smithi135 ceph-mon[32689]: from='mgr.24440 ' entity='mgr.y' cmd='[{"prefix": "mgr fail", "who": "y"}]': finished 2024-09-11T08:14:49.229 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:48 smithi135 ceph-mon[32689]: mgrmap e22: x(active, starting, since 0.00604647s) 2024-09-11T08:14:49.229 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:48 smithi135 ceph-mon[32689]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:14:49.229 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:48 smithi135 ceph-mon[32689]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:14:49.229 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:48 smithi135 ceph-mon[32689]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:14:49.229 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:48 smithi135 ceph-mon[32689]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T08:14:49.229 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:48 smithi135 ceph-mon[32689]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:14:49.229 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:48 smithi135 ceph-mon[32689]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:14:49.229 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:48 smithi135 ceph-mon[32689]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:14:49.229 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:48 smithi135 ceph-mon[32689]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:14:49.229 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:48 smithi135 ceph-mon[32689]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:14:49.229 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:48 smithi135 ceph-mon[32689]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:14:49.229 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:48 smithi135 ceph-mon[32689]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:14:49.230 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:48 smithi135 ceph-mon[32689]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:14:49.230 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:48 smithi135 ceph-mon[32689]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T08:14:49.230 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:48 smithi135 ceph-mon[32689]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T08:14:49.230 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:48 smithi135 ceph-mon[32689]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T08:14:49.230 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:48 smithi135 ceph-mon[32689]: Manager daemon x is now available 2024-09-11T08:14:49.230 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:48 smithi135 ceph-mon[32689]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:14:49.230 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:48 smithi135 ceph-mon[32689]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:14:49.230 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:48 smithi135 ceph-mon[32689]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-11T08:14:49.230 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:48 smithi135 ceph-mon[32689]: from='mgr.24466 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-11T08:14:49.230 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:48 smithi135 ceph-mon[32689]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-11T08:14:49.230 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:48 smithi135 ceph-mon[32689]: from='mgr.24466 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-11T08:14:49.233 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:14:48.864+0000 7f6c5fc34ec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T08:14:49.233 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:14:48.927+0000 7f6c5fc34ec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T08:14:49.233 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:49 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:14:49.108+0000 7f6c5fc34ec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T08:14:49.234 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:14:49.234 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:14:49.234 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-mon[28992]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "mgr fail", "who": "y"}]: dispatch 2024-09-11T08:14:49.234 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' cmd=[{"prefix": "mgr fail", "who": "y"}]: dispatch 2024-09-11T08:14:49.234 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-mon[28992]: Activating manager daemon x 2024-09-11T08:14:49.234 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-mon[28992]: osdmap e64: 8 total, 8 up, 8 in 2024-09-11T08:14:49.235 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-mon[28992]: from='mgr.24440 ' entity='mgr.y' cmd='[{"prefix": "mgr fail", "who": "y"}]': finished 2024-09-11T08:14:49.235 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-mon[28992]: mgrmap e22: x(active, starting, since 0.00604647s) 2024-09-11T08:14:49.235 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-mon[28992]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:14:49.235 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-mon[28992]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:14:49.235 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-mon[28992]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:14:49.235 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-mon[28992]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T08:14:49.235 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-mon[28992]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:14:49.235 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-mon[28992]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:14:49.235 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-mon[28992]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:14:49.235 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-mon[28992]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:14:49.235 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-mon[28992]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:14:49.235 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-mon[28992]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:14:49.235 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-mon[28992]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:14:49.236 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-mon[28992]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:14:49.236 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-mon[28992]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T08:14:49.236 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-mon[28992]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T08:14:49.236 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-mon[28992]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T08:14:49.236 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-mon[28992]: Manager daemon x is now available 2024-09-11T08:14:49.236 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-mon[28992]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:14:49.236 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-mon[28992]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:14:49.236 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-mon[28992]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-11T08:14:49.236 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-mon[28992]: from='mgr.24466 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-11T08:14:49.236 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-mon[28992]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-11T08:14:49.236 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-mon[28992]: from='mgr.24466 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-11T08:14:49.237 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:14:49.237 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' 2024-09-11T08:14:49.237 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-mon[39726]: from='mgr.24440 172.21.15.122:0/1498099677' entity='mgr.y' cmd=[{"prefix": "mgr fail", "who": "y"}]: dispatch 2024-09-11T08:14:49.237 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' cmd=[{"prefix": "mgr fail", "who": "y"}]: dispatch 2024-09-11T08:14:49.237 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-mon[39726]: Activating manager daemon x 2024-09-11T08:14:49.237 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-mon[39726]: osdmap e64: 8 total, 8 up, 8 in 2024-09-11T08:14:49.237 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-mon[39726]: from='mgr.24440 ' entity='mgr.y' cmd='[{"prefix": "mgr fail", "who": "y"}]': finished 2024-09-11T08:14:49.237 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-mon[39726]: mgrmap e22: x(active, starting, since 0.00604647s) 2024-09-11T08:14:49.237 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-mon[39726]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:14:49.237 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-mon[39726]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:14:49.237 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-mon[39726]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:14:49.237 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-mon[39726]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T08:14:49.238 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-mon[39726]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:14:49.238 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-mon[39726]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:14:49.238 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-mon[39726]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:14:49.238 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-mon[39726]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:14:49.238 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-mon[39726]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:14:49.238 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-mon[39726]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:14:49.238 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-mon[39726]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:14:49.238 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-mon[39726]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:14:49.238 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-mon[39726]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T08:14:49.238 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-mon[39726]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T08:14:49.238 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-mon[39726]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T08:14:49.238 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-mon[39726]: Manager daemon x is now available 2024-09-11T08:14:49.238 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-mon[39726]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:14:49.239 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-mon[39726]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:14:49.239 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-mon[39726]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-11T08:14:49.239 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-mon[39726]: from='mgr.24466 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-11T08:14:49.239 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-mon[39726]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-11T08:14:49.239 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:48 smithi122 ceph-mon[39726]: from='mgr.24466 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-11T08:14:49.978 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:14:49 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:14:49.653+0000 7f31e6288640 -1 log_channel(cephadm) log [ERR] : [11/Sep/2024:08:14:49] ENGINE Error in HTTPServer.serve 2024-09-11T08:14:49.978 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:14:49 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: Traceback (most recent call last): 2024-09-11T08:14:49.978 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:14:49 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T08:14:49.978 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:14:49 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: self._connections.run(self.expiration_interval) 2024-09-11T08:14:49.978 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:14:49 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T08:14:49.979 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:14:49 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: self._run(expiration_interval) 2024-09-11T08:14:49.979 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:14:49 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T08:14:49.979 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:14:49 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T08:14:49.979 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:14:49 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T08:14:49.979 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:14:49 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T08:14:49.979 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:14:49 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T08:14:49.979 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:14:49 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: s = self.context.wrap_socket( 2024-09-11T08:14:49.979 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:14:49 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T08:14:49.979 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:14:49 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: return self.sslsocket_class._create( 2024-09-11T08:14:49.979 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:14:49 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T08:14:49.979 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:14:49 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: self.do_handshake() 2024-09-11T08:14:49.979 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:14:49 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T08:14:49.979 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:14:49 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: self._sslobj.do_handshake() 2024-09-11T08:14:49.980 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:14:49 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T08:14:49.980 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:14:49 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:14:50.429 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:50 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:14:50.011+0000 7f6c5fc34ec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T08:14:50.429 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:50 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:14:50.088+0000 7f6c5fc34ec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T08:14:50.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:50 smithi135 ceph-mon[32689]: mgrmap e23: x(active, since 1.36861s) 2024-09-11T08:14:50.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:50 smithi135 ceph-mon[32689]: [11/Sep/2024:08:14:49] ENGINE Bus STARTING 2024-09-11T08:14:50.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:50 smithi135 ceph-mon[32689]: from='mgr.24466 ' entity='mgr.x' 2024-09-11T08:14:50.733 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:50 smithi122 ceph-mon[39726]: mgrmap e23: x(active, since 1.36861s) 2024-09-11T08:14:50.733 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:50 smithi122 ceph-mon[39726]: [11/Sep/2024:08:14:49] ENGINE Bus STARTING 2024-09-11T08:14:50.733 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:50 smithi122 ceph-mon[39726]: from='mgr.24466 ' entity='mgr.x' 2024-09-11T08:14:50.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:50 smithi122 ceph-mon[28992]: mgrmap e23: x(active, since 1.36861s) 2024-09-11T08:14:50.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:50 smithi122 ceph-mon[28992]: [11/Sep/2024:08:14:49] ENGINE Bus STARTING 2024-09-11T08:14:50.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:50 smithi122 ceph-mon[28992]: from='mgr.24466 ' entity='mgr.x' 2024-09-11T08:14:51.137 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:50 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:14:50.829+0000 7f6c5fc34ec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T08:14:51.137 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:50 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:14:50.951+0000 7f6c5fc34ec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T08:14:51.138 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:51 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:14:51.076+0000 7f6c5fc34ec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T08:14:51.138 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:51 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:14:51.137+0000 7f6c5fc34ec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T08:14:51.390 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:51 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:14:51.265+0000 7f6c5fc34ec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T08:14:51.390 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:51 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:14:51.333+0000 7f6c5fc34ec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T08:14:51.391 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:51 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:14:51.390+0000 7f6c5fc34ec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T08:14:51.422 INFO:teuthology.orchestra.run.smithi122.stderr:++ ceph auth get-key mgr.y 2024-09-11T08:14:51.676 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:51 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:14:51.537+0000 7f6c5fc34ec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T08:14:51.677 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:51 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:14:51.606+0000 7f6c5fc34ec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T08:14:51.677 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:51 smithi122 ceph-mon[39726]: [11/Sep/2024:08:14:49] ENGINE Serving on https://172.21.15.135:7150 2024-09-11T08:14:51.677 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:51 smithi122 ceph-mon[39726]: [11/Sep/2024:08:14:49] ENGINE Bus STARTED 2024-09-11T08:14:51.677 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:51 smithi122 ceph-mon[39726]: [11/Sep/2024:08:14:49] ENGINE Error in HTTPServer.serve 2024-09-11T08:14:51.677 INFO:journalctl@ceph.mon.c.smithi122.stdout: Traceback (most recent call last): 2024-09-11T08:14:51.677 INFO:journalctl@ceph.mon.c.smithi122.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T08:14:51.677 INFO:journalctl@ceph.mon.c.smithi122.stdout: self._connections.run(self.expiration_interval) 2024-09-11T08:14:51.678 INFO:journalctl@ceph.mon.c.smithi122.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T08:14:51.678 INFO:journalctl@ceph.mon.c.smithi122.stdout: self._run(expiration_interval) 2024-09-11T08:14:51.678 INFO:journalctl@ceph.mon.c.smithi122.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T08:14:51.678 INFO:journalctl@ceph.mon.c.smithi122.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T08:14:51.678 INFO:journalctl@ceph.mon.c.smithi122.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T08:14:51.678 INFO:journalctl@ceph.mon.c.smithi122.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T08:14:51.678 INFO:journalctl@ceph.mon.c.smithi122.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T08:14:51.678 INFO:journalctl@ceph.mon.c.smithi122.stdout: s = self.context.wrap_socket( 2024-09-11T08:14:51.678 INFO:journalctl@ceph.mon.c.smithi122.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T08:14:51.678 INFO:journalctl@ceph.mon.c.smithi122.stdout: return self.sslsocket_class._create( 2024-09-11T08:14:51.678 INFO:journalctl@ceph.mon.c.smithi122.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T08:14:51.678 INFO:journalctl@ceph.mon.c.smithi122.stdout: self.do_handshake() 2024-09-11T08:14:51.678 INFO:journalctl@ceph.mon.c.smithi122.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T08:14:51.679 INFO:journalctl@ceph.mon.c.smithi122.stdout: self._sslobj.do_handshake() 2024-09-11T08:14:51.679 INFO:journalctl@ceph.mon.c.smithi122.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T08:14:51.679 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:51 smithi122 ceph-mon[39726]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:14:51.679 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:51 smithi122 ceph-mon[28992]: [11/Sep/2024:08:14:49] ENGINE Serving on https://172.21.15.135:7150 2024-09-11T08:14:51.679 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:51 smithi122 ceph-mon[28992]: [11/Sep/2024:08:14:49] ENGINE Bus STARTED 2024-09-11T08:14:51.679 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:51 smithi122 ceph-mon[28992]: [11/Sep/2024:08:14:49] ENGINE Error in HTTPServer.serve 2024-09-11T08:14:51.679 INFO:journalctl@ceph.mon.a.smithi122.stdout: Traceback (most recent call last): 2024-09-11T08:14:51.679 INFO:journalctl@ceph.mon.a.smithi122.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T08:14:51.679 INFO:journalctl@ceph.mon.a.smithi122.stdout: self._connections.run(self.expiration_interval) 2024-09-11T08:14:51.679 INFO:journalctl@ceph.mon.a.smithi122.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T08:14:51.679 INFO:journalctl@ceph.mon.a.smithi122.stdout: self._run(expiration_interval) 2024-09-11T08:14:51.680 INFO:journalctl@ceph.mon.a.smithi122.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T08:14:51.680 INFO:journalctl@ceph.mon.a.smithi122.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T08:14:51.680 INFO:journalctl@ceph.mon.a.smithi122.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T08:14:51.680 INFO:journalctl@ceph.mon.a.smithi122.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T08:14:51.680 INFO:journalctl@ceph.mon.a.smithi122.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T08:14:51.680 INFO:journalctl@ceph.mon.a.smithi122.stdout: s = self.context.wrap_socket( 2024-09-11T08:14:51.680 INFO:journalctl@ceph.mon.a.smithi122.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T08:14:51.680 INFO:journalctl@ceph.mon.a.smithi122.stdout: return self.sslsocket_class._create( 2024-09-11T08:14:51.680 INFO:journalctl@ceph.mon.a.smithi122.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T08:14:51.680 INFO:journalctl@ceph.mon.a.smithi122.stdout: self.do_handshake() 2024-09-11T08:14:51.680 INFO:journalctl@ceph.mon.a.smithi122.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T08:14:51.680 INFO:journalctl@ceph.mon.a.smithi122.stdout: self._sslobj.do_handshake() 2024-09-11T08:14:51.680 INFO:journalctl@ceph.mon.a.smithi122.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T08:14:51.681 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:51 smithi122 ceph-mon[28992]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:14:51.726 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:51 smithi135 ceph-mon[32689]: [11/Sep/2024:08:14:49] ENGINE Serving on https://172.21.15.135:7150 2024-09-11T08:14:51.726 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:51 smithi135 ceph-mon[32689]: [11/Sep/2024:08:14:49] ENGINE Bus STARTED 2024-09-11T08:14:51.726 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:51 smithi135 ceph-mon[32689]: [11/Sep/2024:08:14:49] ENGINE Error in HTTPServer.serve 2024-09-11T08:14:51.727 INFO:journalctl@ceph.mon.b.smithi135.stdout: Traceback (most recent call last): 2024-09-11T08:14:51.727 INFO:journalctl@ceph.mon.b.smithi135.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T08:14:51.727 INFO:journalctl@ceph.mon.b.smithi135.stdout: self._connections.run(self.expiration_interval) 2024-09-11T08:14:51.727 INFO:journalctl@ceph.mon.b.smithi135.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T08:14:51.727 INFO:journalctl@ceph.mon.b.smithi135.stdout: self._run(expiration_interval) 2024-09-11T08:14:51.727 INFO:journalctl@ceph.mon.b.smithi135.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T08:14:51.727 INFO:journalctl@ceph.mon.b.smithi135.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T08:14:51.727 INFO:journalctl@ceph.mon.b.smithi135.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T08:14:51.727 INFO:journalctl@ceph.mon.b.smithi135.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T08:14:51.727 INFO:journalctl@ceph.mon.b.smithi135.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T08:14:51.727 INFO:journalctl@ceph.mon.b.smithi135.stdout: s = self.context.wrap_socket( 2024-09-11T08:14:51.727 INFO:journalctl@ceph.mon.b.smithi135.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T08:14:51.728 INFO:journalctl@ceph.mon.b.smithi135.stdout: return self.sslsocket_class._create( 2024-09-11T08:14:51.728 INFO:journalctl@ceph.mon.b.smithi135.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T08:14:51.728 INFO:journalctl@ceph.mon.b.smithi135.stdout: self.do_handshake() 2024-09-11T08:14:51.728 INFO:journalctl@ceph.mon.b.smithi135.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T08:14:51.728 INFO:journalctl@ceph.mon.b.smithi135.stdout: self._sslobj.do_handshake() 2024-09-11T08:14:51.728 INFO:journalctl@ceph.mon.b.smithi135.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T08:14:51.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:51 smithi135 ceph-mon[32689]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:14:51.742 INFO:teuthology.orchestra.run.smithi122.stdout:rotating key for mgr.x 2024-09-11T08:14:51.742 INFO:teuthology.orchestra.run.smithi122.stderr:+ NK=AQB2UeFmNhw+HRAAKVRWVO1xvEqyrwUbquaDMw== 2024-09-11T08:14:51.742 INFO:teuthology.orchestra.run.smithi122.stderr:+ '[' AQBZTuFml29WBxAAvLofKoNFoe1uzS337O1tEA== == AQB2UeFmNhw+HRAAKVRWVO1xvEqyrwUbquaDMw== ']' 2024-09-11T08:14:51.742 INFO:teuthology.orchestra.run.smithi122.stderr:+ for f in osd.0 osd.1 osd.2 osd.3 osd.4 osd.5 osd.6 osd.7 mgr.y mgr.x 2024-09-11T08:14:51.742 INFO:teuthology.orchestra.run.smithi122.stderr:+ echo 'rotating key for mgr.x' 2024-09-11T08:14:51.742 INFO:teuthology.orchestra.run.smithi122.stderr:++ ceph auth get-key mgr.x 2024-09-11T08:14:51.984 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:51 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:14:51.676+0000 7f6c5fc34ec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T08:14:51.984 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:51 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:14:51.763+0000 7f6c5fc34ec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T08:14:51.984 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:51 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:14:51.842+0000 7f6c5fc34ec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T08:14:51.984 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:51 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:14:51.917+0000 7f6c5fc34ec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T08:14:52.062 INFO:teuthology.orchestra.run.smithi122.stderr:+ K=AQDDTuFmQuj+HxAA6MQZZbc45+rnOAKhNOBDQg== 2024-09-11T08:14:52.062 INFO:teuthology.orchestra.run.smithi122.stderr:+ NK=AQDDTuFmQuj+HxAA6MQZZbc45+rnOAKhNOBDQg== 2024-09-11T08:14:52.062 INFO:teuthology.orchestra.run.smithi122.stderr:+ ceph orch daemon rotate-key mgr.x 2024-09-11T08:14:52.343 INFO:teuthology.orchestra.run.smithi122.stdout:Scheduled to rotate-key mgr.x on host 'smithi135' 2024-09-11T08:14:52.354 INFO:teuthology.orchestra.run.smithi122.stderr:+ '[' AQDDTuFmQuj+HxAA6MQZZbc45+rnOAKhNOBDQg== == AQDDTuFmQuj+HxAA6MQZZbc45+rnOAKhNOBDQg== ']' 2024-09-11T08:14:52.354 INFO:teuthology.orchestra.run.smithi122.stderr:+ sleep 5 2024-09-11T08:14:52.695 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:52 smithi122 ceph-mon[39726]: mgrmap e24: x(active, since 3s) 2024-09-11T08:14:52.696 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:52 smithi122 ceph-mon[39726]: from='mgr.24466 ' entity='mgr.x' 2024-09-11T08:14:52.696 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:52 smithi122 ceph-mon[39726]: from='mgr.24466 ' entity='mgr.x' 2024-09-11T08:14:52.696 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:52 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/1974806723' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.y"}]: dispatch 2024-09-11T08:14:52.696 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:52 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/3186153235' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.x"}]: dispatch 2024-09-11T08:14:52.697 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:52 smithi122 ceph-mon[39726]: from='mgr.24466 ' entity='mgr.x' 2024-09-11T08:14:52.697 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:52 smithi122 ceph-mon[39726]: from='mgr.24466 ' entity='mgr.x' 2024-09-11T08:14:52.697 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:52 smithi122 ceph-mon[39726]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:14:52.697 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:52 smithi122 ceph-mon[39726]: from='mgr.24466 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:14:52.698 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:52 smithi122 ceph-mon[39726]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:14:52.698 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:52 smithi122 ceph-mon[39726]: from='mgr.24466 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:14:52.698 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:52 smithi122 ceph-mon[39726]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:14:52.698 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:52 smithi122 ceph-mon[39726]: from='mgr.24466 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:14:52.698 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:52 smithi122 ceph-mon[39726]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:14:52.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:52 smithi122 ceph-mon[39726]: from='mgr.24466 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:14:52.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:52 smithi122 ceph-mon[39726]: from='mgr.24466 ' entity='mgr.x' 2024-09-11T08:14:52.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:52 smithi122 ceph-mon[39726]: from='mgr.24466 ' entity='mgr.x' 2024-09-11T08:14:52.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:52 smithi122 ceph-mon[39726]: from='mgr.24466 ' entity='mgr.x' 2024-09-11T08:14:52.700 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:52 smithi122 ceph-mon[28992]: mgrmap e24: x(active, since 3s) 2024-09-11T08:14:52.700 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:52 smithi122 ceph-mon[28992]: from='mgr.24466 ' entity='mgr.x' 2024-09-11T08:14:52.700 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:52 smithi122 ceph-mon[28992]: from='mgr.24466 ' entity='mgr.x' 2024-09-11T08:14:52.701 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:52 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/1974806723' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.y"}]: dispatch 2024-09-11T08:14:52.701 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:52 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/3186153235' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.x"}]: dispatch 2024-09-11T08:14:52.701 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:52 smithi122 ceph-mon[28992]: from='mgr.24466 ' entity='mgr.x' 2024-09-11T08:14:52.701 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:52 smithi122 ceph-mon[28992]: from='mgr.24466 ' entity='mgr.x' 2024-09-11T08:14:52.701 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:52 smithi122 ceph-mon[28992]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:14:52.702 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:52 smithi122 ceph-mon[28992]: from='mgr.24466 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:14:52.702 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:52 smithi122 ceph-mon[28992]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:14:52.702 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:52 smithi122 ceph-mon[28992]: from='mgr.24466 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:14:52.702 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:52 smithi122 ceph-mon[28992]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:14:52.703 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:52 smithi122 ceph-mon[28992]: from='mgr.24466 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:14:52.703 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:52 smithi122 ceph-mon[28992]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:14:52.703 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:52 smithi122 ceph-mon[28992]: from='mgr.24466 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:14:52.703 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:52 smithi122 ceph-mon[28992]: from='mgr.24466 ' entity='mgr.x' 2024-09-11T08:14:52.704 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:52 smithi122 ceph-mon[28992]: from='mgr.24466 ' entity='mgr.x' 2024-09-11T08:14:52.704 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:52 smithi122 ceph-mon[28992]: from='mgr.24466 ' entity='mgr.x' 2024-09-11T08:14:52.728 INFO:journalctl@ceph.iscsi.iscsi.a.smithi135.stdout:Sep 11 08:14:52 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[59799]: debug there is no tcmu-runner data available 2024-09-11T08:14:52.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:52 smithi135 ceph-mon[32689]: mgrmap e24: x(active, since 3s) 2024-09-11T08:14:52.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:52 smithi135 ceph-mon[32689]: from='mgr.24466 ' entity='mgr.x' 2024-09-11T08:14:52.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:52 smithi135 ceph-mon[32689]: from='mgr.24466 ' entity='mgr.x' 2024-09-11T08:14:52.729 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:52 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/1974806723' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.y"}]: dispatch 2024-09-11T08:14:52.729 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:52 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/3186153235' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.x"}]: dispatch 2024-09-11T08:14:52.729 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:52 smithi135 ceph-mon[32689]: from='mgr.24466 ' entity='mgr.x' 2024-09-11T08:14:52.729 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:52 smithi135 ceph-mon[32689]: from='mgr.24466 ' entity='mgr.x' 2024-09-11T08:14:52.729 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:52 smithi135 ceph-mon[32689]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:14:52.729 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:52 smithi135 ceph-mon[32689]: from='mgr.24466 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:14:52.730 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:52 smithi135 ceph-mon[32689]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:14:52.730 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:52 smithi135 ceph-mon[32689]: from='mgr.24466 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:14:52.730 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:52 smithi135 ceph-mon[32689]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:14:52.730 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:52 smithi135 ceph-mon[32689]: from='mgr.24466 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:14:52.730 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:52 smithi135 ceph-mon[32689]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:14:52.730 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:52 smithi135 ceph-mon[32689]: from='mgr.24466 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:14:52.731 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:52 smithi135 ceph-mon[32689]: from='mgr.24466 ' entity='mgr.x' 2024-09-11T08:14:52.731 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:52 smithi135 ceph-mon[32689]: from='mgr.24466 ' entity='mgr.x' 2024-09-11T08:14:52.731 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:52 smithi135 ceph-mon[32689]: from='mgr.24466 ' entity='mgr.x' 2024-09-11T08:14:52.984 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:52 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:14:52.694+0000 7f6c5fc34ec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T08:14:52.984 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:52 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: [11/Sep/2024:08:14:52] ENGINE Bus STARTING 2024-09-11T08:14:52.984 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:52 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: CherryPy Checker: 2024-09-11T08:14:52.984 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:52 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: The Application mounted at '' has an empty config. 2024-09-11T08:14:52.984 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:52 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: 2024-09-11T08:14:52.984 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:52 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: [11/Sep/2024:08:14:52] ENGINE Serving on http://:::9283 2024-09-11T08:14:52.984 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:52 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: [11/Sep/2024:08:14:52] ENGINE Bus STARTED 2024-09-11T08:14:53.549 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:53 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[29217]: ::ffff:172.21.15.135 - - [11/Sep/2024:08:14:53] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:14:53.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:53 smithi135 ceph-mon[32689]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:14:53.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:53 smithi135 ceph-mon[32689]: Adjusting osd_memory_target on smithi122 to 3247M 2024-09-11T08:14:53.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:53 smithi135 ceph-mon[32689]: from='client.25286 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "mgr.x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:14:53.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:53 smithi135 ceph-mon[32689]: Schedule rotate-key daemon mgr.x 2024-09-11T08:14:53.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:53 smithi135 ceph-mon[32689]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:14:53.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:53 smithi135 ceph-mon[32689]: Standby manager daemon y started 2024-09-11T08:14:53.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:53 smithi135 ceph-mon[32689]: from='mgr.? 172.21.15.122:0/3862444874' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-11T08:14:53.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:53 smithi135 ceph-mon[32689]: from='mgr.? 172.21.15.122:0/3862444874' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T08:14:53.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:53 smithi135 ceph-mon[32689]: from='mgr.? 172.21.15.122:0/3862444874' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-11T08:14:53.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:53 smithi135 ceph-mon[32689]: from='mgr.? 172.21.15.122:0/3862444874' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T08:14:53.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:53 smithi135 ceph-mon[32689]: from='mgr.24466 ' entity='mgr.x' 2024-09-11T08:14:53.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:53 smithi135 ceph-mon[32689]: from='mgr.24466 ' entity='mgr.x' 2024-09-11T08:14:53.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:53 smithi135 ceph-mon[32689]: from='mgr.24466 ' entity='mgr.x' 2024-09-11T08:14:53.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:53 smithi135 ceph-mon[32689]: from='mgr.24466 ' entity='mgr.x' 2024-09-11T08:14:53.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:53 smithi135 ceph-mon[32689]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:14:53.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:53 smithi135 ceph-mon[32689]: from='mgr.24466 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:14:53.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:53 smithi135 ceph-mon[32689]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:14:53.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:53 smithi135 ceph-mon[32689]: from='mgr.24466 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:14:53.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:53 smithi135 ceph-mon[32689]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:14:53.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:53 smithi135 ceph-mon[32689]: from='mgr.24466 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:14:53.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:53 smithi135 ceph-mon[32689]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:14:53.980 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:53 smithi135 ceph-mon[32689]: from='mgr.24466 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:14:53.980 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:53 smithi135 ceph-mon[32689]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:14:53.980 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:53 smithi135 ceph-mon[32689]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:14:53.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:53 smithi122 ceph-mon[39726]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:14:53.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:53 smithi122 ceph-mon[39726]: Adjusting osd_memory_target on smithi122 to 3247M 2024-09-11T08:14:53.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:53 smithi122 ceph-mon[39726]: from='client.25286 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "mgr.x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:14:53.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:53 smithi122 ceph-mon[39726]: Schedule rotate-key daemon mgr.x 2024-09-11T08:14:53.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:53 smithi122 ceph-mon[39726]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:14:53.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:53 smithi122 ceph-mon[39726]: Standby manager daemon y started 2024-09-11T08:14:53.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:53 smithi122 ceph-mon[39726]: from='mgr.? 172.21.15.122:0/3862444874' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-11T08:14:53.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:53 smithi122 ceph-mon[39726]: from='mgr.? 172.21.15.122:0/3862444874' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T08:14:53.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:53 smithi122 ceph-mon[39726]: from='mgr.? 172.21.15.122:0/3862444874' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-11T08:14:53.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:53 smithi122 ceph-mon[39726]: from='mgr.? 172.21.15.122:0/3862444874' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T08:14:53.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:53 smithi122 ceph-mon[39726]: from='mgr.24466 ' entity='mgr.x' 2024-09-11T08:14:53.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:53 smithi122 ceph-mon[39726]: from='mgr.24466 ' entity='mgr.x' 2024-09-11T08:14:53.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:53 smithi122 ceph-mon[39726]: from='mgr.24466 ' entity='mgr.x' 2024-09-11T08:14:53.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:53 smithi122 ceph-mon[39726]: from='mgr.24466 ' entity='mgr.x' 2024-09-11T08:14:53.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:53 smithi122 ceph-mon[39726]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:14:53.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:53 smithi122 ceph-mon[39726]: from='mgr.24466 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:14:53.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:53 smithi122 ceph-mon[39726]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:14:53.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:53 smithi122 ceph-mon[39726]: from='mgr.24466 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:14:53.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:53 smithi122 ceph-mon[39726]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:14:53.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:53 smithi122 ceph-mon[39726]: from='mgr.24466 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:14:53.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:53 smithi122 ceph-mon[39726]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:14:53.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:53 smithi122 ceph-mon[39726]: from='mgr.24466 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:14:53.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:53 smithi122 ceph-mon[39726]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:14:53.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:53 smithi122 ceph-mon[39726]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:14:53.986 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:53 smithi122 ceph-mon[28992]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:14:53.986 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:53 smithi122 ceph-mon[28992]: Adjusting osd_memory_target on smithi122 to 3247M 2024-09-11T08:14:53.986 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:53 smithi122 ceph-mon[28992]: from='client.25286 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "mgr.x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:14:53.986 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:53 smithi122 ceph-mon[28992]: Schedule rotate-key daemon mgr.x 2024-09-11T08:14:53.986 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:53 smithi122 ceph-mon[28992]: from='client.24421 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:14:53.986 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:53 smithi122 ceph-mon[28992]: Standby manager daemon y started 2024-09-11T08:14:53.986 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:53 smithi122 ceph-mon[28992]: from='mgr.? 172.21.15.122:0/3862444874' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-11T08:14:53.986 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:53 smithi122 ceph-mon[28992]: from='mgr.? 172.21.15.122:0/3862444874' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T08:14:53.986 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:53 smithi122 ceph-mon[28992]: from='mgr.? 172.21.15.122:0/3862444874' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-11T08:14:53.986 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:53 smithi122 ceph-mon[28992]: from='mgr.? 172.21.15.122:0/3862444874' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T08:14:53.986 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:53 smithi122 ceph-mon[28992]: from='mgr.24466 ' entity='mgr.x' 2024-09-11T08:14:53.987 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:53 smithi122 ceph-mon[28992]: from='mgr.24466 ' entity='mgr.x' 2024-09-11T08:14:53.987 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:53 smithi122 ceph-mon[28992]: from='mgr.24466 ' entity='mgr.x' 2024-09-11T08:14:53.987 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:53 smithi122 ceph-mon[28992]: from='mgr.24466 ' entity='mgr.x' 2024-09-11T08:14:53.987 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:53 smithi122 ceph-mon[28992]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:14:53.987 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:53 smithi122 ceph-mon[28992]: from='mgr.24466 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:14:53.987 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:53 smithi122 ceph-mon[28992]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:14:53.987 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:53 smithi122 ceph-mon[28992]: from='mgr.24466 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:14:53.987 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:53 smithi122 ceph-mon[28992]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:14:53.987 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:53 smithi122 ceph-mon[28992]: from='mgr.24466 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:14:53.987 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:53 smithi122 ceph-mon[28992]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:14:53.987 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:53 smithi122 ceph-mon[28992]: from='mgr.24466 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:14:53.987 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:53 smithi122 ceph-mon[28992]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:14:53.987 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:53 smithi122 ceph-mon[28992]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:14:54.887 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:54 smithi122 ceph-mon[39726]: Adjusting osd_memory_target on smithi135 to 429.8M 2024-09-11T08:14:54.888 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:54 smithi122 ceph-mon[39726]: Unable to set osd_memory_target on smithi135 to 450693836: error parsing value: Value '450693836' is below minimum 939524096 2024-09-11T08:14:54.888 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:54 smithi122 ceph-mon[39726]: Updating smithi122:/etc/ceph/ceph.conf 2024-09-11T08:14:54.888 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:54 smithi122 ceph-mon[39726]: Updating smithi135:/etc/ceph/ceph.conf 2024-09-11T08:14:54.888 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:54 smithi122 ceph-mon[39726]: mgrmap e25: x(active, since 5s), standbys: y 2024-09-11T08:14:54.888 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:54 smithi122 ceph-mon[39726]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T08:14:54.888 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:54 smithi122 ceph-mon[39726]: Updating smithi122:/var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:14:54.888 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:54 smithi122 ceph-mon[39726]: Updating smithi135:/var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:14:54.888 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:54 smithi122 ceph-mon[39726]: from='mgr.24466 ' entity='mgr.x' 2024-09-11T08:14:54.888 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:54 smithi122 ceph-mon[39726]: from='mgr.24466 ' entity='mgr.x' 2024-09-11T08:14:54.888 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:54 smithi122 ceph-mon[39726]: from='mgr.24466 ' entity='mgr.x' 2024-09-11T08:14:54.888 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:54 smithi122 ceph-mon[39726]: from='mgr.24466 ' entity='mgr.x' 2024-09-11T08:14:54.888 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:54 smithi122 ceph-mon[39726]: from='mgr.24466 ' entity='mgr.x' 2024-09-11T08:14:54.889 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:54 smithi122 ceph-mon[39726]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]: dispatch 2024-09-11T08:14:54.889 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:54 smithi122 ceph-mon[39726]: from='mgr.24466 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]: dispatch 2024-09-11T08:14:54.889 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:54 smithi122 ceph-mon[39726]: from='mgr.24466 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]': finished 2024-09-11T08:14:54.889 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:54 smithi122 ceph-mon[39726]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T08:14:54.889 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:54 smithi122 ceph-mon[39726]: from='mgr.24466 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T08:14:54.889 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:54 smithi122 ceph-mon[39726]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T08:14:54.889 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:54 smithi122 ceph-mon[39726]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:14:54.889 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:54 smithi122 ceph-mon[28992]: Adjusting osd_memory_target on smithi135 to 429.8M 2024-09-11T08:14:54.889 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:54 smithi122 ceph-mon[28992]: Unable to set osd_memory_target on smithi135 to 450693836: error parsing value: Value '450693836' is below minimum 939524096 2024-09-11T08:14:54.889 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:54 smithi122 ceph-mon[28992]: Updating smithi122:/etc/ceph/ceph.conf 2024-09-11T08:14:54.889 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:54 smithi122 ceph-mon[28992]: Updating smithi135:/etc/ceph/ceph.conf 2024-09-11T08:14:54.889 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:54 smithi122 ceph-mon[28992]: mgrmap e25: x(active, since 5s), standbys: y 2024-09-11T08:14:54.890 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:54 smithi122 ceph-mon[28992]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T08:14:54.890 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:54 smithi122 ceph-mon[28992]: Updating smithi122:/var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:14:54.890 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:54 smithi122 ceph-mon[28992]: Updating smithi135:/var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:14:54.890 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:54 smithi122 ceph-mon[28992]: from='mgr.24466 ' entity='mgr.x' 2024-09-11T08:14:54.890 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:54 smithi122 ceph-mon[28992]: from='mgr.24466 ' entity='mgr.x' 2024-09-11T08:14:54.890 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:54 smithi122 ceph-mon[28992]: from='mgr.24466 ' entity='mgr.x' 2024-09-11T08:14:54.890 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:54 smithi122 ceph-mon[28992]: from='mgr.24466 ' entity='mgr.x' 2024-09-11T08:14:54.890 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:54 smithi122 ceph-mon[28992]: from='mgr.24466 ' entity='mgr.x' 2024-09-11T08:14:54.890 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:54 smithi122 ceph-mon[28992]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]: dispatch 2024-09-11T08:14:54.890 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:54 smithi122 ceph-mon[28992]: from='mgr.24466 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]: dispatch 2024-09-11T08:14:54.890 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:54 smithi122 ceph-mon[28992]: from='mgr.24466 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]': finished 2024-09-11T08:14:54.890 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:54 smithi122 ceph-mon[28992]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T08:14:54.890 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:54 smithi122 ceph-mon[28992]: from='mgr.24466 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T08:14:54.890 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:54 smithi122 ceph-mon[28992]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T08:14:54.891 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:54 smithi122 ceph-mon[28992]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:14:54.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:54 smithi135 ceph-mon[32689]: Adjusting osd_memory_target on smithi135 to 429.8M 2024-09-11T08:14:54.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:54 smithi135 ceph-mon[32689]: Unable to set osd_memory_target on smithi135 to 450693836: error parsing value: Value '450693836' is below minimum 939524096 2024-09-11T08:14:54.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:54 smithi135 ceph-mon[32689]: Updating smithi122:/etc/ceph/ceph.conf 2024-09-11T08:14:54.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:54 smithi135 ceph-mon[32689]: Updating smithi135:/etc/ceph/ceph.conf 2024-09-11T08:14:54.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:54 smithi135 ceph-mon[32689]: mgrmap e25: x(active, since 5s), standbys: y 2024-09-11T08:14:54.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:54 smithi135 ceph-mon[32689]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T08:14:54.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:54 smithi135 ceph-mon[32689]: Updating smithi122:/var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:14:54.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:54 smithi135 ceph-mon[32689]: Updating smithi135:/var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:14:54.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:54 smithi135 ceph-mon[32689]: from='mgr.24466 ' entity='mgr.x' 2024-09-11T08:14:54.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:54 smithi135 ceph-mon[32689]: from='mgr.24466 ' entity='mgr.x' 2024-09-11T08:14:54.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:54 smithi135 ceph-mon[32689]: from='mgr.24466 ' entity='mgr.x' 2024-09-11T08:14:54.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:54 smithi135 ceph-mon[32689]: from='mgr.24466 ' entity='mgr.x' 2024-09-11T08:14:54.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:54 smithi135 ceph-mon[32689]: from='mgr.24466 ' entity='mgr.x' 2024-09-11T08:14:54.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:54 smithi135 ceph-mon[32689]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]: dispatch 2024-09-11T08:14:54.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:54 smithi135 ceph-mon[32689]: from='mgr.24466 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]: dispatch 2024-09-11T08:14:54.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:54 smithi135 ceph-mon[32689]: from='mgr.24466 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]': finished 2024-09-11T08:14:54.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:54 smithi135 ceph-mon[32689]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T08:14:54.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:54 smithi135 ceph-mon[32689]: from='mgr.24466 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T08:14:54.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:54 smithi135 ceph-mon[32689]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T08:14:54.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:54 smithi135 ceph-mon[32689]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:14:55.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:55 smithi135 ceph-mon[32689]: Updating smithi135:/etc/ceph/ceph.client.admin.keyring 2024-09-11T08:14:55.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:55 smithi135 ceph-mon[32689]: Updating smithi122:/etc/ceph/ceph.client.admin.keyring 2024-09-11T08:14:55.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:55 smithi135 ceph-mon[32689]: Updating smithi122:/var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T08:14:55.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:55 smithi135 ceph-mon[32689]: Updating smithi135:/var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T08:14:55.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:55 smithi135 ceph-mon[32689]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:14:55.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:55 smithi135 ceph-mon[32689]: Rotating authentication key for mgr.y 2024-09-11T08:14:55.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:55 smithi135 ceph-mon[32689]: Reconfiguring daemon mgr.y on smithi122 2024-09-11T08:14:55.983 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:55 smithi122 ceph-mon[39726]: Updating smithi135:/etc/ceph/ceph.client.admin.keyring 2024-09-11T08:14:55.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:55 smithi122 ceph-mon[39726]: Updating smithi122:/etc/ceph/ceph.client.admin.keyring 2024-09-11T08:14:55.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:55 smithi122 ceph-mon[39726]: Updating smithi122:/var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T08:14:55.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:55 smithi122 ceph-mon[39726]: Updating smithi135:/var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T08:14:55.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:55 smithi122 ceph-mon[39726]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:14:55.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:55 smithi122 ceph-mon[39726]: Rotating authentication key for mgr.y 2024-09-11T08:14:55.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:55 smithi122 ceph-mon[39726]: Reconfiguring daemon mgr.y on smithi122 2024-09-11T08:14:55.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:55 smithi122 ceph-mon[28992]: Updating smithi135:/etc/ceph/ceph.client.admin.keyring 2024-09-11T08:14:55.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:55 smithi122 ceph-mon[28992]: Updating smithi122:/etc/ceph/ceph.client.admin.keyring 2024-09-11T08:14:55.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:55 smithi122 ceph-mon[28992]: Updating smithi122:/var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T08:14:55.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:55 smithi122 ceph-mon[28992]: Updating smithi135:/var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T08:14:55.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:55 smithi122 ceph-mon[28992]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:14:55.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:55 smithi122 ceph-mon[28992]: Rotating authentication key for mgr.y 2024-09-11T08:14:55.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:55 smithi122 ceph-mon[28992]: Reconfiguring daemon mgr.y on smithi122 2024-09-11T08:14:56.686 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:56 smithi122 systemd[1]: Stopping Ceph mgr.y for f1d75850-7013-11ef-bcea-c7b262605968... 2024-09-11T08:14:56.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:56 smithi122 ceph-mon[28992]: from='mgr.24466 ' entity='mgr.x' 2024-09-11T08:14:56.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:56 smithi122 ceph-mon[28992]: from='mgr.24466 ' entity='mgr.x' 2024-09-11T08:14:56.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:56 smithi122 ceph-mon[28992]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 27 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-11T08:14:56.984 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:56 smithi122 podman[80841]: 2024-09-11 08:14:56.685137992 +0000 UTC m=+0.134366141 container died e4267cade852a6cfbb3eb68ed1593a40cf129d6b161427850f95232f005cd093 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-11T08:14:56.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:56 smithi122 ceph-mon[39726]: from='mgr.24466 ' entity='mgr.x' 2024-09-11T08:14:56.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:56 smithi122 ceph-mon[39726]: from='mgr.24466 ' entity='mgr.x' 2024-09-11T08:14:56.985 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:56 smithi122 ceph-mon[39726]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 27 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-11T08:14:57.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:56 smithi135 ceph-mon[32689]: from='mgr.24466 ' entity='mgr.x' 2024-09-11T08:14:57.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:56 smithi135 ceph-mon[32689]: from='mgr.24466 ' entity='mgr.x' 2024-09-11T08:14:57.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:56 smithi135 ceph-mon[32689]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 27 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-11T08:14:57.331 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:57 smithi122 podman[80841]: 2024-09-11 08:14:57.080492381 +0000 UTC m=+0.529720530 container cleanup e4267cade852a6cfbb3eb68ed1593a40cf129d6b161427850f95232f005cd093 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-11T08:14:57.332 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:57 smithi122 bash[80841]: ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y 2024-09-11T08:14:57.356 INFO:teuthology.orchestra.run.smithi122.stderr:++ ceph auth get-key mgr.x 2024-09-11T08:14:57.694 INFO:teuthology.orchestra.run.smithi122.stderr:+ NK=AQDDTuFmQuj+HxAA6MQZZbc45+rnOAKhNOBDQg== 2024-09-11T08:14:57.694 INFO:teuthology.orchestra.run.smithi122.stderr:+ '[' AQDDTuFmQuj+HxAA6MQZZbc45+rnOAKhNOBDQg== == AQDDTuFmQuj+HxAA6MQZZbc45+rnOAKhNOBDQg== ']' 2024-09-11T08:14:57.695 INFO:teuthology.orchestra.run.smithi122.stderr:+ sleep 5 2024-09-11T08:14:57.695 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:57 smithi122 podman[80854]: 2024-09-11 08:14:57.33171189 +0000 UTC m=+0.645443075 container remove e4267cade852a6cfbb3eb68ed1593a40cf129d6b161427850f95232f005cd093 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2) 2024-09-11T08:14:57.695 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:57 smithi122 systemd[1]: ceph-f1d75850-7013-11ef-bcea-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-09-11T08:14:57.695 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:57 smithi122 systemd[1]: ceph-f1d75850-7013-11ef-bcea-c7b262605968@mgr.y.service: Failed with result 'exit-code'. 2024-09-11T08:14:57.695 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:57 smithi122 systemd[1]: Stopped Ceph mgr.y for f1d75850-7013-11ef-bcea-c7b262605968. 2024-09-11T08:14:57.695 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:57 smithi122 systemd[1]: ceph-f1d75850-7013-11ef-bcea-c7b262605968@mgr.y.service: Consumed 43.162s CPU time. 2024-09-11T08:14:57.695 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:57 smithi122 systemd[1]: Starting Ceph mgr.y for f1d75850-7013-11ef-bcea-c7b262605968... 2024-09-11T08:14:57.983 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:57 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/1146421028' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.x"}]: dispatch 2024-09-11T08:14:57.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:57 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/1146421028' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.x"}]: dispatch 2024-09-11T08:14:57.984 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:57 smithi122 podman[80966]: 2024-09-11 08:14:57.639964268 +0000 UTC m=+0.019325457 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 2024-09-11T08:14:57.984 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:57 smithi122 podman[80966]: 2024-09-11 08:14:57.749589223 +0000 UTC m=+0.128950446 container create f7fb71753baac7bc3f0456a7610c5143847b6782c6cec3c58c94caf8b86729fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.schema-version=1.0) 2024-09-11T08:14:58.227 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:57 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/1146421028' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.x"}]: dispatch 2024-09-11T08:14:58.228 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:14:57 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: ::ffff:172.21.15.135 - - [11/Sep/2024:08:14:57] "GET /metrics HTTP/1.1" 200 34930 "" "Prometheus/2.43.0" 2024-09-11T08:14:58.265 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:58 smithi122 podman[80966]: 2024-09-11 08:14:58.019129932 +0000 UTC m=+0.398491114 container init f7fb71753baac7bc3f0456a7610c5143847b6782c6cec3c58c94caf8b86729fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-11T08:14:58.266 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:58 smithi122 podman[80966]: 2024-09-11 08:14:58.022068004 +0000 UTC m=+0.401429182 container start f7fb71753baac7bc3f0456a7610c5143847b6782c6cec3c58c94caf8b86729fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, io.buildah.version=1.37.2, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-11T08:14:58.266 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:58 smithi122 ceph-mgr[80988]: -- 172.21.15.122:0/267556810 <== mon.2 v2:172.21.15.122:3301/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x558700648340 con 0x558700654800 2024-09-11T08:14:58.266 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:58 smithi122 ceph-mgr[80988]: cephx client: could not set rotating key: decode_decrypt failed. error:bad magic in decode_decrypt, 18347727433554900172 != 18374858748799134293 2024-09-11T08:14:58.266 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:58 smithi122 bash[80966]: f7fb71753baac7bc3f0456a7610c5143847b6782c6cec3c58c94caf8b86729fe 2024-09-11T08:14:58.266 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:58 smithi122 systemd[1]: Started Ceph mgr.y for f1d75850-7013-11ef-bcea-c7b262605968. 2024-09-11T08:14:58.266 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:58 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[80984]: 2024-09-11T08:14:58.142+0000 7fe23cd11ec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T08:14:58.266 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:58 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[80984]: 2024-09-11T08:14:58.199+0000 7fe23cd11ec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T08:14:58.266 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:58 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[80984]: 2024-09-11T08:14:58.265+0000 7fe23cd11ec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T08:14:58.538 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:58 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[80984]: 2024-09-11T08:14:58.395+0000 7fe23cd11ec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T08:14:58.810 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:58 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[80984]: 2024-09-11T08:14:58.537+0000 7fe23cd11ec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T08:14:59.125 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:58 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[80984]: 2024-09-11T08:14:58.809+0000 7fe23cd11ec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T08:14:59.125 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:58 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[80984]: 2024-09-11T08:14:58.871+0000 7fe23cd11ec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T08:14:59.125 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:59 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[80984]: 2024-09-11T08:14:59.038+0000 7fe23cd11ec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T08:14:59.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:59 smithi135 ceph-mon[32689]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 21 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-11T08:14:59.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:59 smithi135 ceph-mon[32689]: from='mgr.24466 ' entity='mgr.x' 2024-09-11T08:14:59.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:59 smithi135 ceph-mon[32689]: from='mgr.24466 ' entity='mgr.x' 2024-09-11T08:14:59.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:59 smithi135 ceph-mon[32689]: Rotating authentication key for mgr.x 2024-09-11T08:14:59.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:59 smithi135 ceph-mon[32689]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.x", "format": "json"}]: dispatch 2024-09-11T08:14:59.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:59 smithi135 ceph-mon[32689]: from='mgr.24466 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.x", "format": "json"}]: dispatch 2024-09-11T08:14:59.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:59 smithi135 ceph-mon[32689]: from='mgr.24466 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create-pending", "entity": "mgr.x", "format": "json"}]': finished 2024-09-11T08:14:59.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:59 smithi135 ceph-mon[32689]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T08:14:59.478 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:59 smithi135 ceph-mon[32689]: from='mgr.24466 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T08:14:59.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:59 smithi135 ceph-mon[32689]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T08:14:59.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:59 smithi135 ceph-mon[32689]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:14:59.479 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:14:59 smithi135 ceph-mon[32689]: Reconfiguring daemon mgr.x on smithi135 2024-09-11T08:14:59.483 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:59 smithi122 ceph-mon[39726]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 21 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-11T08:14:59.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:59 smithi122 ceph-mon[39726]: from='mgr.24466 ' entity='mgr.x' 2024-09-11T08:14:59.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:59 smithi122 ceph-mon[39726]: from='mgr.24466 ' entity='mgr.x' 2024-09-11T08:14:59.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:59 smithi122 ceph-mon[39726]: Rotating authentication key for mgr.x 2024-09-11T08:14:59.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:59 smithi122 ceph-mon[39726]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.x", "format": "json"}]: dispatch 2024-09-11T08:14:59.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:59 smithi122 ceph-mon[39726]: from='mgr.24466 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.x", "format": "json"}]: dispatch 2024-09-11T08:14:59.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:59 smithi122 ceph-mon[39726]: from='mgr.24466 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create-pending", "entity": "mgr.x", "format": "json"}]': finished 2024-09-11T08:14:59.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:59 smithi122 ceph-mon[39726]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T08:14:59.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:59 smithi122 ceph-mon[39726]: from='mgr.24466 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T08:14:59.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:59 smithi122 ceph-mon[39726]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T08:14:59.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:59 smithi122 ceph-mon[39726]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:14:59.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:14:59 smithi122 ceph-mon[39726]: Reconfiguring daemon mgr.x on smithi135 2024-09-11T08:14:59.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:59 smithi122 ceph-mon[28992]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 21 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-11T08:14:59.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:59 smithi122 ceph-mon[28992]: from='mgr.24466 ' entity='mgr.x' 2024-09-11T08:14:59.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:59 smithi122 ceph-mon[28992]: from='mgr.24466 ' entity='mgr.x' 2024-09-11T08:14:59.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:59 smithi122 ceph-mon[28992]: Rotating authentication key for mgr.x 2024-09-11T08:14:59.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:59 smithi122 ceph-mon[28992]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.x", "format": "json"}]: dispatch 2024-09-11T08:14:59.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:59 smithi122 ceph-mon[28992]: from='mgr.24466 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.x", "format": "json"}]: dispatch 2024-09-11T08:14:59.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:59 smithi122 ceph-mon[28992]: from='mgr.24466 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create-pending", "entity": "mgr.x", "format": "json"}]': finished 2024-09-11T08:14:59.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:59 smithi122 ceph-mon[28992]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T08:14:59.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:59 smithi122 ceph-mon[28992]: from='mgr.24466 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T08:14:59.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:59 smithi122 ceph-mon[28992]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T08:14:59.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:59 smithi122 ceph-mon[28992]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:14:59.485 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:14:59 smithi122 ceph-mon[28992]: Reconfiguring daemon mgr.x on smithi135 2024-09-11T08:14:59.823 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:14:59 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:14:59.673+0000 7f321baf3640 -1 mgr.server reply reply (13) Permission denied access denied: does your client key have mgr caps? See http://docs.ceph.com/en/latest/mgr/administrator/#client-authentication 2024-09-11T08:14:59.823 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:14:59 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:14:59.675+0000 7f32192ee640 -1 log_channel(cephadm) log [ERR] : Non-zero return from ['ceph', '-k', '/var/lib/ceph/mgr/ceph-x/keyring', '-n', 'mgr.x', 'tell', 'mgr.x', 'rotate-key', '-i', '-']: 2024-09-11T08:14:59.669+0000 7fbcf6ffd640 1 -- 172.21.15.135:0/80686983 <== mon.1 v2:172.21.15.135:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x7fbce4003710 con 0x7fbcf8104710 2024-09-11T08:14:59.823 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:14:59 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:14:59.669+0000 7fbcf6ffd640 0 cephx client: could not set rotating key: decode_decrypt failed. error:bad magic in decode_decrypt, 6717969834744339244 != 18374858748799134293 2024-09-11T08:14:59.823 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:14:59 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:14:59.669+0000 7fbcfe4a3640 1 -- 172.21.15.135:0/80686983 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbcf8104710 msgr2=0x7fbcf8106af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:14:59.824 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:14:59 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:14:59.669+0000 7fbcfe4a3640 1 --2- 172.21.15.135:0/80686983 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbcf8104710 0x7fbcf8106af0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fbce0004950 tx=0x7fbce0009da0 comp rx=0 tx=0).stop 2024-09-11T08:14:59.824 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:14:59 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:14:59.669+0000 7fbcfe4a3640 1 -- 172.21.15.135:0/80686983 shutdown_connections 2024-09-11T08:14:59.824 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:14:59 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:14:59.669+0000 7fbcfe4a3640 1 --2- 172.21.15.135:0/80686983 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbcf8109b70 0x7fbcf810c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:14:59.824 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:14:59 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:14:59.669+0000 7fbcfe4a3640 1 --2- 172.21.15.135:0/80686983 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbcf81070c0 0x7fbcf81094e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:14:59.824 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:14:59 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:14:59.669+0000 7fbcfe4a3640 1 --2- 172.21.15.135:0/80686983 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbcf8104710 0x7fbcf8106af0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:14:59.824 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:14:59 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:14:59.669+0000 7fbcfe4a3640 1 -- 172.21.15.135:0/80686983 >> 172.21.15.135:0/80686983 conn(0x7fbcf806aa40 msgr2=0x7fbcf806ce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:14:59.824 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:14:59 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:14:59.669+0000 7fbcfe4a3640 1 -- 172.21.15.135:0/80686983 shutdown_connections 2024-09-11T08:14:59.824 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:14:59 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:14:59.669+0000 7fbcfe4a3640 1 -- 172.21.15.135:0/80686983 wait complete. 2024-09-11T08:14:59.824 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:14:59 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:14:59.670+0000 7fbcfe4a3640 1 Processor -- start 2024-09-11T08:14:59.824 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:14:59 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:14:59.670+0000 7fbcfe4a3640 1 -- start start 2024-09-11T08:14:59.824 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:14:59 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:14:59.670+0000 7fbcfe4a3640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbcf8104710 0x7fbcf81142d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:14:59.824 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:14:59 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:14:59.670+0000 7fbcfe4a3640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbcf81070c0 0x7fbcf8114810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:14:59.825 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:14:59 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:14:59.670+0000 7fbcfe4a3640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbcf8109b70 0x7fbcf811b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:14:59.825 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:14:59 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:14:59.670+0000 7fbcfe4a3640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf811be30 con 0x7fbcf8104710 2024-09-11T08:14:59.825 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:14:59 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:14:59.670+0000 7fbcfe4a3640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf811bfa0 con 0x7fbcf81070c0 2024-09-11T08:14:59.825 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:14:59 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:14:59.670+0000 7fbcfe4a3640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf811c160 con 0x7fbcf8109b70 2024-09-11T08:14:59.825 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:14:59 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:14:59.670+0000 7fbcf77fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbcf81070c0 0x7fbcf8114810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:14:59.825 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:14:59 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:14:59.670+0000 7fbcf77fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbcf81070c0 0x7fbcf8114810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:49092/0 (socket says 172.21.15.135:49092) 2024-09-11T08:14:59.825 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:14:59 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:14:59.670+0000 7fbcf77fe640 1 -- 172.21.15.135:0/1153541798 learned_addr learned my addr 172.21.15.135:0/1153541798 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-09-11T08:14:59.825 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:14:59 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:14:59.670+0000 7fbcf7fff640 1 --2- 172.21.15.135:0/1153541798 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbcf8104710 0x7fbcf81142d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:14:59.825 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:14:59 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:14:59.670+0000 7fbcfca40640 1 --2- 172.21.15.135:0/1153541798 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbcf8109b70 0x7fbcf811b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:14:59.825 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:14:59 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:14:59.670+0000 7fbcf77fe640 1 -- 172.21.15.135:0/1153541798 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbcf8109b70 msgr2=0x7fbcf811b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:14:59.825 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:14:59 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:14:59.670+0000 7fbcf77fe640 1 --2- 172.21.15.135:0/1153541798 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbcf8109b70 0x7fbcf811b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:14:59.826 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:14:59 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:14:59.670+0000 7fbcf77fe640 1 -- 172.21.15.135:0/1153541798 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbcf8104710 msgr2=0x7fbcf81142d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:14:59.826 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:14:59 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:14:59.670+0000 7fbcf77fe640 1 --2- 172.21.15.135:0/1153541798 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbcf8104710 0x7fbcf81142d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T08:14:59.826 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:14:59 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:14:59.670+0000 7fbcf77fe640 1 -- 172.21.15.135:0/1153541798 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbce0005040 con 0x7fbcf81070c0 2024-09-11T08:14:59.826 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:14:59 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:14:59.670+0000 7fbcf77fe640 1 --2- 172.21.15.135:0/1153541798 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbcf81070c0 0x7fbcf8114810 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fbce8002ad0 tx=0x7fbce8004600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:14:59.826 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:14:59 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:14:59.671+0000 7fbcf4ff9640 1 -- 172.21.15.135:0/1153541798 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce8007c30 con 0x7fbcf81070c0 2024-09-11T08:14:59.826 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:14:59 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:14:59.671+0000 7fbcf4ff9640 1 -- 172.21.15.135:0/1153541798 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- auth(proto 2 2 bytes epoch 0) v1 -- 0x7fbce40032d0 con 0x7fbcf81070c0 2024-09-11T08:14:59.826 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:14:59 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:14:59.671+0000 7fbcf4ff9640 1 -- 172.21.15.135:0/1153541798 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(36 keys) v1 ==== 1701+0+0 (secure 0 0 0) 0x7fbce800f040 con 0x7fbcf81070c0 2024-09-11T08:14:59.826 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:14:59 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:14:59.671+0000 7fbcfe4a3640 1 -- 172.21.15.135:0/1153541798 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcf811c440 con 0x7fbcf81070c0 2024-09-11T08:14:59.826 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:14:59 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:14:59.671+0000 7fbcf4ff9640 1 -- 172.21.15.135:0/1153541798 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce8013890 con 0x7fbcf81070c0 2024-09-11T08:14:59.826 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:14:59 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:14:59.671+0000 7fbcfe4a3640 1 -- 172.21.15.135:0/1153541798 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcf811c930 con 0x7fbcf81070c0 2024-09-11T08:14:59.827 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:14:59 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:14:59.671+0000 7fbcf4ff9640 1 -- 172.21.15.135:0/1153541798 <== mon.1 v2:172.21.15.135:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x7fbcf811c930 con 0x7fbcf81070c0 2024-09-11T08:14:59.827 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:14:59 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:14:59.671+0000 7fbcf4ff9640 0 cephx client: could not set rotating key: decode_decrypt failed. error:bad magic in decode_decrypt, 6717969834744339244 != 18374858748799134293 2024-09-11T08:14:59.827 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:14:59 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:14:59.672+0000 7fbcf4ff9640 1 -- 172.21.15.135:0/1153541798 <== mon.1 v2:172.21.15.135:3300/0 5 ==== mgrmap(e 25) v1 ==== 90839+0+0 (secure 0 0 0) 0x7fbce8023020 con 0x7fbcf81070c0 2024-09-11T08:14:59.827 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:14:59 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:14:59.672+0000 7fbcf4ff9640 1 --2- 172.21.15.135:0/1153541798 >> [v2:172.21.15.135:6832/4165606104,v1:172.21.15.135:6833/4165606104] conn(0x7fbce406f210 0x7fbce40716c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T08:14:59.827 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:14:59 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:14:59.672+0000 7fbcf4ff9640 1 -- 172.21.15.135:0/1153541798 --> [v2:172.21.15.135:6832/4165606104,v1:172.21.15.135:6833/4165606104] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbce4071dd0 con 0x7fbce406f210 2024-09-11T08:14:59.827 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:14:59 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:14:59.672+0000 7fbcf4ff9640 1 -- 172.21.15.135:0/1153541798 <== mon.1 v2:172.21.15.135:3300/0 6 ==== osd_map(64..64 src has 1..64) v4 ==== 7791+0+0 (secure 0 0 0) 0x7fbce808f190 con 0x7fbcf81070c0 2024-09-11T08:14:59.827 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:14:59 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:14:59.673+0000 7fbcf7fff640 1 --2- 172.21.15.135:0/1153541798 >> [v2:172.21.15.135:6832/4165606104,v1:172.21.15.135:6833/4165606104] conn(0x7fbce406f210 0x7fbce40716c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T08:14:59.827 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:14:59 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:14:59.673+0000 7fbcf7fff640 1 --2- 172.21.15.135:0/1153541798 >> [v2:172.21.15.135:6832/4165606104,v1:172.21.15.135:6833/4165606104] conn(0x7fbce406f210 0x7fbce40716c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fbce00047f0 tx=0x7fbce0009da0 comp rx=0 tx=0).ready entity=mgr.24466 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T08:14:59.827 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:14:59 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:14:59.673+0000 7fbcf4ff9640 1 -- 172.21.15.135:0/1153541798 <== mgr.24466 v2:172.21.15.135:6832/4165606104 1 ==== command_reply(tid 0: -13 access denied: does your client key have mgr caps? See http://docs.ceph.com/en/latest/mgr/administrator/#client-authentication) v1 ==== 134+0+0 (secure 0 0 0) 0x7fbce4071dd0 con 0x7fbce406f210 2024-09-11T08:14:59.827 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:14:59 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: Error EACCES: problem getting command descriptions from mgr.x 2024-09-11T08:14:59.827 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:14:59 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:14:59.827 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:14:59 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:14:59.688+0000 7f323e71a640 -1 mgr handle_mgr_map I was active but no longer am 2024-09-11T08:14:59.827 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:14:59 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: ignoring --setuser ceph since I am not root 2024-09-11T08:14:59.828 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:14:59 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: ignoring --setgroup ceph since I am not root 2024-09-11T08:14:59.828 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:14:59 smithi135 ceph-mgr[34558]: -- 172.21.15.135:0/1342292310 <== mon.1 v2:172.21.15.135:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55db47bd2340 con 0x55db47bde800 2024-09-11T08:14:59.828 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:14:59 smithi135 ceph-mgr[34558]: cephx client: could not set rotating key: decode_decrypt failed. error:bad magic in decode_decrypt, 6717969834744339244 != 18374858748799134293 2024-09-11T08:14:59.828 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:14:59 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:14:59.821+0000 7f55ca004ec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T08:14:59.976 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:59 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[80984]: 2024-09-11T08:14:59.912+0000 7fe23cd11ec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T08:14:59.976 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:14:59 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[80984]: 2024-09-11T08:14:59.975+0000 7fe23cd11ec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T08:15:00.204 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:14:59 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:14:59.876+0000 7f55ca004ec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T08:15:00.204 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:14:59 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:14:59.941+0000 7f55ca004ec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T08:15:00.204 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:15:00 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:15:00.065+0000 7f55ca004ec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T08:15:00.465 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:15:00 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:15:00.202+0000 7f55ca004ec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T08:15:00.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:15:00 smithi135 ceph-mon[32689]: from='mgr.24466 ' entity='mgr.x' 2024-09-11T08:15:00.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:15:00 smithi135 ceph-mon[32689]: from='mgr.24466 ' entity='mgr.x' 2024-09-11T08:15:00.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:15:00 smithi135 ceph-mon[32689]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "mgr fail", "who": "x"}]: dispatch 2024-09-11T08:15:00.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:15:00 smithi135 ceph-mon[32689]: from='mgr.24466 ' entity='mgr.x' cmd=[{"prefix": "mgr fail", "who": "x"}]: dispatch 2024-09-11T08:15:00.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:15:00 smithi135 ceph-mon[32689]: Activating manager daemon y 2024-09-11T08:15:00.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:15:00 smithi135 ceph-mon[32689]: osdmap e65: 8 total, 8 up, 8 in 2024-09-11T08:15:00.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:15:00 smithi135 ceph-mon[32689]: from='mgr.24466 ' entity='mgr.x' cmd='[{"prefix": "mgr fail", "who": "x"}]': finished 2024-09-11T08:15:00.728 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:15:00 smithi135 ceph-mon[32689]: mgrmap e26: y(active, starting, since 0.0092396s) 2024-09-11T08:15:00.729 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:15:00 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:15:00.465+0000 7f55ca004ec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T08:15:00.729 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:15:00 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:15:00.527+0000 7f55ca004ec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T08:15:00.729 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:15:00 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:15:00.702+0000 7f55ca004ec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T08:15:00.917 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:15:00 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[80984]: 2024-09-11T08:15:00.686+0000 7fe23cd11ec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T08:15:00.917 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:15:00 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[80984]: 2024-09-11T08:15:00.798+0000 7fe23cd11ec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T08:15:00.917 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:15:00 smithi122 ceph-mon[39726]: from='mgr.24466 ' entity='mgr.x' 2024-09-11T08:15:00.918 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:15:00 smithi122 ceph-mon[39726]: from='mgr.24466 ' entity='mgr.x' 2024-09-11T08:15:00.918 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:15:00 smithi122 ceph-mon[39726]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "mgr fail", "who": "x"}]: dispatch 2024-09-11T08:15:00.918 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:15:00 smithi122 ceph-mon[39726]: from='mgr.24466 ' entity='mgr.x' cmd=[{"prefix": "mgr fail", "who": "x"}]: dispatch 2024-09-11T08:15:00.918 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:15:00 smithi122 ceph-mon[39726]: Activating manager daemon y 2024-09-11T08:15:00.918 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:15:00 smithi122 ceph-mon[39726]: osdmap e65: 8 total, 8 up, 8 in 2024-09-11T08:15:00.918 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:15:00 smithi122 ceph-mon[39726]: from='mgr.24466 ' entity='mgr.x' cmd='[{"prefix": "mgr fail", "who": "x"}]': finished 2024-09-11T08:15:00.918 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:15:00 smithi122 ceph-mon[39726]: mgrmap e26: y(active, starting, since 0.0092396s) 2024-09-11T08:15:00.918 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:15:00 smithi122 ceph-mon[28992]: from='mgr.24466 ' entity='mgr.x' 2024-09-11T08:15:00.918 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:15:00 smithi122 ceph-mon[28992]: from='mgr.24466 ' entity='mgr.x' 2024-09-11T08:15:00.918 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:15:00 smithi122 ceph-mon[28992]: from='mgr.24466 172.21.15.135:0/2259543084' entity='mgr.x' cmd=[{"prefix": "mgr fail", "who": "x"}]: dispatch 2024-09-11T08:15:00.918 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:15:00 smithi122 ceph-mon[28992]: from='mgr.24466 ' entity='mgr.x' cmd=[{"prefix": "mgr fail", "who": "x"}]: dispatch 2024-09-11T08:15:00.919 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:15:00 smithi122 ceph-mon[28992]: Activating manager daemon y 2024-09-11T08:15:00.919 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:15:00 smithi122 ceph-mon[28992]: osdmap e65: 8 total, 8 up, 8 in 2024-09-11T08:15:00.919 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:15:00 smithi122 ceph-mon[28992]: from='mgr.24466 ' entity='mgr.x' cmd='[{"prefix": "mgr fail", "who": "x"}]': finished 2024-09-11T08:15:00.919 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:15:00 smithi122 ceph-mon[28992]: mgrmap e26: y(active, starting, since 0.0092396s) 2024-09-11T08:15:01.224 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:15:00 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[80984]: 2024-09-11T08:15:00.917+0000 7fe23cd11ec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T08:15:01.224 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:15:00 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[80984]: 2024-09-11T08:15:00.976+0000 7fe23cd11ec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T08:15:01.224 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:15:01 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[80984]: 2024-09-11T08:15:01.101+0000 7fe23cd11ec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T08:15:01.224 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:15:01 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[80984]: 2024-09-11T08:15:01.167+0000 7fe23cd11ec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T08:15:01.483 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:15:01 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[80984]: 2024-09-11T08:15:01.223+0000 7fe23cd11ec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T08:15:01.484 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:15:01 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[80984]: 2024-09-11T08:15:01.357+0000 7fe23cd11ec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T08:15:01.484 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:15:01 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[80984]: 2024-09-11T08:15:01.419+0000 7fe23cd11ec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T08:15:01.728 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:15:01 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:15:01.580+0000 7f55ca004ec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T08:15:01.728 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:15:01 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:15:01.644+0000 7f55ca004ec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T08:15:01.983 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:15:01 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[80984]: 2024-09-11T08:15:01.484+0000 7fe23cd11ec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T08:15:01.984 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:15:01 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[80984]: 2024-09-11T08:15:01.565+0000 7fe23cd11ec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T08:15:01.984 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:15:01 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[80984]: 2024-09-11T08:15:01.623+0000 7fe23cd11ec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T08:15:01.984 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:15:01 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[80984]: 2024-09-11T08:15:01.680+0000 7fe23cd11ec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T08:15:02.644 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:15:02 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:15:02.347+0000 7f55ca004ec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T08:15:02.644 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:15:02 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:15:02.460+0000 7f55ca004ec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T08:15:02.644 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:15:02 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:15:02.583+0000 7f55ca004ec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T08:15:02.644 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:15:02 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:15:02.642+0000 7f55ca004ec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T08:15:02.697 INFO:teuthology.orchestra.run.smithi122.stderr:++ ceph auth get-key mgr.x 2024-09-11T08:15:02.697 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:15:02 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[80984]: 2024-09-11T08:15:02.426+0000 7fe23cd11ec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T08:15:02.697 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:15:02 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[80984]: [11/Sep/2024:08:15:02] ENGINE Bus STARTING 2024-09-11T08:15:02.697 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:15:02 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[80984]: CherryPy Checker: 2024-09-11T08:15:02.698 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:15:02 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[80984]: The Application mounted at '' has an empty config. 2024-09-11T08:15:02.698 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:15:02 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[80984]: 2024-09-11T08:15:02.698 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:15:02 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[80984]: [11/Sep/2024:08:15:02] ENGINE Serving on http://:::9283 2024-09-11T08:15:02.698 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:15:02 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[80984]: [11/Sep/2024:08:15:02] ENGINE Bus STARTED 2024-09-11T08:15:02.698 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:15:02 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[80984]: [11/Sep/2024:08:15:02] ENGINE Bus STOPPING 2024-09-11T08:15:02.895 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:15:02 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:15:02.768+0000 7f55ca004ec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T08:15:02.896 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:15:02 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:15:02.836+0000 7f55ca004ec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T08:15:02.896 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:15:02 smithi135 ceph-mon[32689]: Active manager daemon y restarted 2024-09-11T08:15:02.896 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:15:02 smithi135 ceph-mon[32689]: Activating manager daemon y 2024-09-11T08:15:02.896 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:15:02 smithi135 ceph-mon[32689]: osdmap e66: 8 total, 8 up, 8 in 2024-09-11T08:15:02.896 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:15:02 smithi135 ceph-mon[32689]: mgrmap e27: y(active, starting, since 0.00612729s) 2024-09-11T08:15:02.896 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:15:02 smithi135 ceph-mon[32689]: from='mgr.? 172.21.15.122:0/1819989557' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-11T08:15:02.896 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:15:02 smithi135 ceph-mon[32689]: from='mgr.15399 172.21.15.122:0/1819989557' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-11T08:15:02.896 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:15:02 smithi135 ceph-mon[32689]: from='mgr.15399 172.21.15.122:0/1819989557' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:15:02.896 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:15:02 smithi135 ceph-mon[32689]: from='mgr.15399 172.21.15.122:0/1819989557' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:15:02.896 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:15:02 smithi135 ceph-mon[32689]: from='mgr.15399 172.21.15.122:0/1819989557' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:15:02.897 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:15:02 smithi135 ceph-mon[32689]: from='mgr.15399 172.21.15.122:0/1819989557' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T08:15:02.897 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:15:02 smithi135 ceph-mon[32689]: from='mgr.15399 172.21.15.122:0/1819989557' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:15:02.897 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:15:02 smithi135 ceph-mon[32689]: from='mgr.15399 172.21.15.122:0/1819989557' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:15:02.897 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:15:02 smithi135 ceph-mon[32689]: from='mgr.15399 172.21.15.122:0/1819989557' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:15:02.897 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:15:02 smithi135 ceph-mon[32689]: from='mgr.15399 172.21.15.122:0/1819989557' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:15:02.897 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:15:02 smithi135 ceph-mon[32689]: from='mgr.15399 172.21.15.122:0/1819989557' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:15:02.897 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:15:02 smithi135 ceph-mon[32689]: from='mgr.15399 172.21.15.122:0/1819989557' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:15:02.897 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:15:02 smithi135 ceph-mon[32689]: from='mgr.15399 172.21.15.122:0/1819989557' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:15:02.897 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:15:02 smithi135 ceph-mon[32689]: from='mgr.15399 172.21.15.122:0/1819989557' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:15:02.897 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:15:02 smithi135 ceph-mon[32689]: from='mgr.15399 172.21.15.122:0/1819989557' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T08:15:02.897 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:15:02 smithi135 ceph-mon[32689]: from='mgr.15399 172.21.15.122:0/1819989557' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T08:15:02.897 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:15:02 smithi135 ceph-mon[32689]: from='mgr.15399 172.21.15.122:0/1819989557' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T08:15:02.898 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:15:02 smithi135 ceph-mon[32689]: from='mgr.15399 172.21.15.122:0/1819989557' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T08:15:02.898 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:15:02 smithi135 ceph-mon[32689]: from='mgr.15399 172.21.15.122:0/1819989557' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-11T08:15:02.898 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:15:02 smithi135 ceph-mon[32689]: from='mgr.15399 172.21.15.122:0/1819989557' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T08:15:02.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:15:02 smithi122 ceph-mon[28992]: Active manager daemon y restarted 2024-09-11T08:15:02.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:15:02 smithi122 ceph-mon[28992]: Activating manager daemon y 2024-09-11T08:15:02.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:15:02 smithi122 ceph-mon[28992]: osdmap e66: 8 total, 8 up, 8 in 2024-09-11T08:15:02.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:15:02 smithi122 ceph-mon[28992]: mgrmap e27: y(active, starting, since 0.00612729s) 2024-09-11T08:15:02.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:15:02 smithi122 ceph-mon[28992]: from='mgr.? 172.21.15.122:0/1819989557' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-11T08:15:02.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:15:02 smithi122 ceph-mon[28992]: from='mgr.15399 172.21.15.122:0/1819989557' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-11T08:15:02.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:15:02 smithi122 ceph-mon[28992]: from='mgr.15399 172.21.15.122:0/1819989557' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:15:02.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:15:02 smithi122 ceph-mon[28992]: from='mgr.15399 172.21.15.122:0/1819989557' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:15:02.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:15:02 smithi122 ceph-mon[28992]: from='mgr.15399 172.21.15.122:0/1819989557' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:15:02.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:15:02 smithi122 ceph-mon[28992]: from='mgr.15399 172.21.15.122:0/1819989557' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T08:15:02.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:15:02 smithi122 ceph-mon[28992]: from='mgr.15399 172.21.15.122:0/1819989557' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:15:02.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:15:02 smithi122 ceph-mon[28992]: from='mgr.15399 172.21.15.122:0/1819989557' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:15:02.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:15:02 smithi122 ceph-mon[28992]: from='mgr.15399 172.21.15.122:0/1819989557' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:15:02.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:15:02 smithi122 ceph-mon[28992]: from='mgr.15399 172.21.15.122:0/1819989557' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:15:02.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:15:02 smithi122 ceph-mon[28992]: from='mgr.15399 172.21.15.122:0/1819989557' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:15:02.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:15:02 smithi122 ceph-mon[28992]: from='mgr.15399 172.21.15.122:0/1819989557' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:15:02.985 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:15:02 smithi122 ceph-mon[28992]: from='mgr.15399 172.21.15.122:0/1819989557' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:15:02.986 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:15:02 smithi122 ceph-mon[28992]: from='mgr.15399 172.21.15.122:0/1819989557' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:15:02.986 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:15:02 smithi122 ceph-mon[28992]: from='mgr.15399 172.21.15.122:0/1819989557' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T08:15:02.986 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:15:02 smithi122 ceph-mon[28992]: from='mgr.15399 172.21.15.122:0/1819989557' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T08:15:02.986 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:15:02 smithi122 ceph-mon[28992]: from='mgr.15399 172.21.15.122:0/1819989557' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T08:15:02.986 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:15:02 smithi122 ceph-mon[28992]: from='mgr.15399 172.21.15.122:0/1819989557' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T08:15:02.986 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:15:02 smithi122 ceph-mon[28992]: from='mgr.15399 172.21.15.122:0/1819989557' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-11T08:15:02.986 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:15:02 smithi122 ceph-mon[28992]: from='mgr.15399 172.21.15.122:0/1819989557' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T08:15:02.986 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:15:02 smithi122 ceph-mon[39726]: Active manager daemon y restarted 2024-09-11T08:15:02.986 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:15:02 smithi122 ceph-mon[39726]: Activating manager daemon y 2024-09-11T08:15:02.986 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:15:02 smithi122 ceph-mon[39726]: osdmap e66: 8 total, 8 up, 8 in 2024-09-11T08:15:02.986 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:15:02 smithi122 ceph-mon[39726]: mgrmap e27: y(active, starting, since 0.00612729s) 2024-09-11T08:15:02.986 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:15:02 smithi122 ceph-mon[39726]: from='mgr.? 172.21.15.122:0/1819989557' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-11T08:15:02.987 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:15:02 smithi122 ceph-mon[39726]: from='mgr.15399 172.21.15.122:0/1819989557' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-11T08:15:02.987 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:15:02 smithi122 ceph-mon[39726]: from='mgr.15399 172.21.15.122:0/1819989557' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:15:02.987 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:15:02 smithi122 ceph-mon[39726]: from='mgr.15399 172.21.15.122:0/1819989557' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:15:02.987 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:15:02 smithi122 ceph-mon[39726]: from='mgr.15399 172.21.15.122:0/1819989557' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:15:02.987 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:15:02 smithi122 ceph-mon[39726]: from='mgr.15399 172.21.15.122:0/1819989557' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T08:15:02.987 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:15:02 smithi122 ceph-mon[39726]: from='mgr.15399 172.21.15.122:0/1819989557' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:15:02.987 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:15:02 smithi122 ceph-mon[39726]: from='mgr.15399 172.21.15.122:0/1819989557' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:15:02.987 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:15:02 smithi122 ceph-mon[39726]: from='mgr.15399 172.21.15.122:0/1819989557' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:15:02.987 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:15:02 smithi122 ceph-mon[39726]: from='mgr.15399 172.21.15.122:0/1819989557' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:15:02.987 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:15:02 smithi122 ceph-mon[39726]: from='mgr.15399 172.21.15.122:0/1819989557' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:15:02.987 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:15:02 smithi122 ceph-mon[39726]: from='mgr.15399 172.21.15.122:0/1819989557' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:15:02.987 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:15:02 smithi122 ceph-mon[39726]: from='mgr.15399 172.21.15.122:0/1819989557' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:15:02.987 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:15:02 smithi122 ceph-mon[39726]: from='mgr.15399 172.21.15.122:0/1819989557' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:15:02.988 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:15:02 smithi122 ceph-mon[39726]: from='mgr.15399 172.21.15.122:0/1819989557' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T08:15:02.988 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:15:02 smithi122 ceph-mon[39726]: from='mgr.15399 172.21.15.122:0/1819989557' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T08:15:02.988 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:15:02 smithi122 ceph-mon[39726]: from='mgr.15399 172.21.15.122:0/1819989557' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T08:15:02.988 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:15:02 smithi122 ceph-mon[39726]: from='mgr.15399 172.21.15.122:0/1819989557' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T08:15:02.988 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:15:02 smithi122 ceph-mon[39726]: from='mgr.15399 172.21.15.122:0/1819989557' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-11T08:15:02.988 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:15:02 smithi122 ceph-mon[39726]: from='mgr.15399 172.21.15.122:0/1819989557' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T08:15:03.000 INFO:teuthology.orchestra.run.smithi122.stderr:+ NK=AQCCUeFmTSgaCBAAvblBpyCzAERTs6/CS+8K7Q== 2024-09-11T08:15:03.000 INFO:teuthology.orchestra.run.smithi122.stderr:+ '[' AQDDTuFmQuj+HxAA6MQZZbc45+rnOAKhNOBDQg== == AQCCUeFmTSgaCBAAvblBpyCzAERTs6/CS+8K7Q== ']' 2024-09-11T08:15:03.159 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:15:02 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:15:02.894+0000 7f55ca004ec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T08:15:03.160 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:15:03 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:15:03.028+0000 7f55ca004ec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T08:15:03.160 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:15:03 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:15:03.093+0000 7f55ca004ec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T08:15:03.160 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:15:03 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:15:03.158+0000 7f55ca004ec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T08:15:03.279 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:15:03 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[80984]: [11/Sep/2024:08:15:03] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-11T08:15:03.280 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:15:03 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[80984]: [11/Sep/2024:08:15:03] ENGINE Bus STOPPED 2024-09-11T08:15:03.280 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:15:03 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[80984]: [11/Sep/2024:08:15:03] ENGINE Bus STARTING 2024-09-11T08:15:03.280 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:15:03 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[80984]: [11/Sep/2024:08:15:03] ENGINE Serving on http://:::9283 2024-09-11T08:15:03.280 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:15:03 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[80984]: [11/Sep/2024:08:15:03] ENGINE Bus STARTED 2024-09-11T08:15:03.478 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:15:03 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:15:03.239+0000 7f55ca004ec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T08:15:03.478 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:15:03 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:15:03.298+0000 7f55ca004ec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T08:15:03.478 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:15:03 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:15:03.358+0000 7f55ca004ec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T08:15:03.682 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:15:03 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[80984]: ::ffff:172.21.15.135 - - [11/Sep/2024:08:15:03] "GET /metrics HTTP/1.1" 200 34927 "" "Prometheus/2.43.0" 2024-09-11T08:15:03.706 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-11T08:15:03.776 INFO:tasks.cephadm:Teardown begin 2024-09-11T08:15:03.777 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T08:15:03.813 DEBUG:teuthology.orchestra.run.smithi135:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T08:15:03.841 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-11T08:15:03.842 DEBUG:teuthology.orchestra.run.smithi122:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-11T08:15:03.872 DEBUG:teuthology.orchestra.run.smithi135:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-11T08:15:03.899 INFO:tasks.cephadm:Stopping all daemons... 2024-09-11T08:15:03.900 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-11T08:15:03.900 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ceph-f1d75850-7013-11ef-bcea-c7b262605968@mon.a 2024-09-11T08:15:03.936 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:15:03 smithi122 ceph-mon[39726]: from='client.? 172.21.15.122:0/2030814188' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.x"}]: dispatch 2024-09-11T08:15:03.936 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:15:03 smithi122 ceph-mon[39726]: Manager daemon y is now available 2024-09-11T08:15:03.936 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:15:03 smithi122 ceph-mon[39726]: from='mgr.15399 172.21.15.122:0/1819989557' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:15:03.936 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:15:03 smithi122 ceph-mon[39726]: from='mgr.15399 172.21.15.122:0/1819989557' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:15:03.936 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:15:03 smithi122 ceph-mon[39726]: from='mgr.15399 172.21.15.122:0/1819989557' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-11T08:15:03.936 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:15:03 smithi122 ceph-mon[39726]: from='mgr.15399 172.21.15.122:0/1819989557' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-11T08:15:03.936 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:15:03 smithi122 ceph-mon[28992]: from='client.? 172.21.15.122:0/2030814188' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.x"}]: dispatch 2024-09-11T08:15:03.937 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:15:03 smithi122 ceph-mon[28992]: Manager daemon y is now available 2024-09-11T08:15:03.937 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:15:03 smithi122 ceph-mon[28992]: from='mgr.15399 172.21.15.122:0/1819989557' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:15:03.937 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:15:03 smithi122 ceph-mon[28992]: from='mgr.15399 172.21.15.122:0/1819989557' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:15:03.937 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:15:03 smithi122 ceph-mon[28992]: from='mgr.15399 172.21.15.122:0/1819989557' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-11T08:15:03.937 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:15:03 smithi122 ceph-mon[28992]: from='mgr.15399 172.21.15.122:0/1819989557' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-11T08:15:04.143 INFO:journalctl@ceph.iscsi.iscsi.a.smithi135.stdout:Sep 11 08:15:03 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[59799]: debug there is no tcmu-runner data available 2024-09-11T08:15:04.143 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:15:03 smithi135 ceph-mon[32689]: from='client.? 172.21.15.122:0/2030814188' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.x"}]: dispatch 2024-09-11T08:15:04.143 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:15:03 smithi135 ceph-mon[32689]: Manager daemon y is now available 2024-09-11T08:15:04.143 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:15:03 smithi135 ceph-mon[32689]: from='mgr.15399 172.21.15.122:0/1819989557' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:15:04.143 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:15:03 smithi135 ceph-mon[32689]: from='mgr.15399 172.21.15.122:0/1819989557' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:15:04.143 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:15:03 smithi135 ceph-mon[32689]: from='mgr.15399 172.21.15.122:0/1819989557' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-11T08:15:04.144 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:15:03 smithi135 ceph-mon[32689]: from='mgr.15399 172.21.15.122:0/1819989557' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-11T08:15:04.234 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:15:03 smithi122 systemd[1]: Stopping Ceph mon.a for f1d75850-7013-11ef-bcea-c7b262605968... 2024-09-11T08:15:04.234 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:15:04 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[80984]: 2024-09-11T08:15:04.160+0000 7fe1df852640 -1 log_channel(cephadm) log [ERR] : [11/Sep/2024:08:15:04] ENGINE Error in HTTPServer.serve 2024-09-11T08:15:04.234 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:15:04 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[80984]: Traceback (most recent call last): 2024-09-11T08:15:04.234 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:15:04 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[80984]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T08:15:04.234 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:15:04 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[80984]: self._connections.run(self.expiration_interval) 2024-09-11T08:15:04.234 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:15:04 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[80984]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T08:15:04.234 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:15:04 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[80984]: self._run(expiration_interval) 2024-09-11T08:15:04.234 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:15:04 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[80984]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T08:15:04.234 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:15:04 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[80984]: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T08:15:04.235 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:15:04 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[80984]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T08:15:04.235 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:15:04 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[80984]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T08:15:04.235 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:15:04 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[80984]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T08:15:04.235 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:15:04 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[80984]: s = self.context.wrap_socket( 2024-09-11T08:15:04.235 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:15:04 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[80984]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T08:15:04.235 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:15:04 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[80984]: return self.sslsocket_class._create( 2024-09-11T08:15:04.235 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:15:04 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[80984]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T08:15:04.235 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:15:04 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[80984]: self.do_handshake() 2024-09-11T08:15:04.235 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:15:04 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[80984]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T08:15:04.235 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:15:04 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[80984]: self._sslobj.do_handshake() 2024-09-11T08:15:04.235 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:15:04 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[80984]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T08:15:04.235 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:15:04 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[80984]: 2024-09-11T08:15:04.480 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:15:04 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:15:04.170+0000 7f55ca004ec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T08:15:04.480 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:15:04 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: [11/Sep/2024:08:15:04] ENGINE Bus STARTING 2024-09-11T08:15:04.481 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:15:04 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: CherryPy Checker: 2024-09-11T08:15:04.481 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:15:04 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: The Application mounted at '' has an empty config. 2024-09-11T08:15:04.481 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:15:04 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: 2024-09-11T08:15:04.481 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:15:04 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: [11/Sep/2024:08:15:04] ENGINE Serving on http://:::9283 2024-09-11T08:15:04.481 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:15:04 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: [11/Sep/2024:08:15:04] ENGINE Bus STARTED 2024-09-11T08:15:04.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:15:04 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mon-a[28988]: 2024-09-11T08:15:04.647+0000 7f6a37a89640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T08:15:04.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:15:04 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mon-a[28988]: 2024-09-11T08:15:04.647+0000 7f6a37a89640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-11T08:15:04.984 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:15:04 smithi122 podman[81506]: 2024-09-11 08:15:04.871919377 +0000 UTC m=+0.356768465 container died b720c3646b61a425df5c73d63fcebd34a3f9998e9a2a85892a54334c1f8dd520 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-11T08:15:04.984 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:15:04 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[80984]: [11/Sep/2024:08:15:04] ENGINE Bus STOPPING 2024-09-11T08:15:04.984 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:15:04 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[80984]: [11/Sep/2024:08:15:04] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-11T08:15:04.984 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:15:04 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[80984]: [11/Sep/2024:08:15:04] ENGINE Bus STOPPED 2024-09-11T08:15:04.985 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:15:04 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[80984]: [11/Sep/2024:08:15:04] ENGINE Bus STARTING 2024-09-11T08:15:04.985 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:15:04 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[80984]: [11/Sep/2024:08:15:04] ENGINE Serving on http://:::9283 2024-09-11T08:15:04.985 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:15:04 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[80984]: [11/Sep/2024:08:15:04] ENGINE Bus STARTED 2024-09-11T08:15:05.483 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:15:05 smithi122 podman[81506]: 2024-09-11 08:15:05.256988403 +0000 UTC m=+0.741837487 container cleanup b720c3646b61a425df5c73d63fcebd34a3f9998e9a2a85892a54334c1f8dd520 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-mon-a, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True) 2024-09-11T08:15:05.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:15:05 smithi122 bash[81506]: ceph-f1d75850-7013-11ef-bcea-c7b262605968-mon-a 2024-09-11T08:15:05.828 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 11 08:15:05 smithi122 podman[81638]: 2024-09-11 08:15:05.508420455 +0000 UTC m=+0.631298850 container remove b720c3646b61a425df5c73d63fcebd34a3f9998e9a2a85892a54334c1f8dd520 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-mon-a, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-11T08:15:06.318 DEBUG:teuthology.orchestra.run.smithi122:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f1d75850-7013-11ef-bcea-c7b262605968@mon.a.service' 2024-09-11T08:15:06.434 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T08:15:06.435 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-11T08:15:06.435 INFO:tasks.cephadm.mon.b:Stopping mon.c... 2024-09-11T08:15:06.435 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ceph-f1d75850-7013-11ef-bcea-c7b262605968@mon.c 2024-09-11T08:15:06.672 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:15:06 smithi122 systemd[1]: Stopping Ceph mon.c for f1d75850-7013-11ef-bcea-c7b262605968... 2024-09-11T08:15:07.483 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:15:07 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mon-c[39722]: 2024-09-11T08:15:07.159+0000 7f26f4857640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T08:15:07.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:15:07 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mon-c[39722]: 2024-09-11T08:15:07.159+0000 7f26f4857640 -1 mon.c@2(peon) e3 *** Got Signal Terminated *** 2024-09-11T08:15:07.484 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:15:07 smithi122 podman[81912]: 2024-09-11 08:15:07.40571718 +0000 UTC m=+0.382346393 container died 1991f5f7e6238204dbb522e5a4d876a498695e9f8a42132c18e997f84bed663a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-mon-c, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-11T08:15:07.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:15:07 smithi122 podman[81912]: 2024-09-11 08:15:07.793845238 +0000 UTC m=+0.770474461 container cleanup 1991f5f7e6238204dbb522e5a4d876a498695e9f8a42132c18e997f84bed663a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-mon-c, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True) 2024-09-11T08:15:07.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 11 08:15:07 smithi122 bash[81912]: ceph-f1d75850-7013-11ef-bcea-c7b262605968-mon-c 2024-09-11T08:15:08.213 DEBUG:teuthology.orchestra.run.smithi122:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f1d75850-7013-11ef-bcea-c7b262605968@mon.c.service' 2024-09-11T08:15:08.227 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:15:07 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x[34554]: ::ffff:172.21.15.135 - - [11/Sep/2024:08:15:07] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:15:08.251 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T08:15:08.251 INFO:tasks.cephadm.mon.b:Stopped mon.c 2024-09-11T08:15:08.251 INFO:tasks.cephadm.mon.b:Stopping mon.b... 2024-09-11T08:15:08.251 DEBUG:teuthology.orchestra.run.smithi135:> sudo systemctl stop ceph-f1d75850-7013-11ef-bcea-c7b262605968@mon.b 2024-09-11T08:15:08.578 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:15:08 smithi135 systemd[1]: Stopping Ceph mon.b for f1d75850-7013-11ef-bcea-c7b262605968... 2024-09-11T08:15:08.928 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:15:08 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mon-b[32672]: 2024-09-11T08:15:08.576+0000 7f9c8961e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T08:15:08.929 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:15:08 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mon-b[32672]: 2024-09-11T08:15:08.576+0000 7f9c8961e640 -1 mon.b@1(peon) e3 *** Got Signal Terminated *** 2024-09-11T08:15:08.929 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:15:08 smithi135 podman[69822]: 2024-09-11 08:15:08.638678408 +0000 UTC m=+0.185996402 container died fc64e10c38b93c28a2d645cf5a0bf09de5bae9dde4aac50fc81c78df280445a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-mon-b, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-11T08:15:09.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:15:09 smithi135 podman[69822]: 2024-09-11 08:15:09.02740176 +0000 UTC m=+0.574719753 container cleanup fc64e10c38b93c28a2d645cf5a0bf09de5bae9dde4aac50fc81c78df280445a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-mon-b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-11T08:15:09.228 INFO:journalctl@ceph.mon.b.smithi135.stdout:Sep 11 08:15:09 smithi135 bash[69822]: ceph-f1d75850-7013-11ef-bcea-c7b262605968-mon-b 2024-09-11T08:15:09.402 DEBUG:teuthology.orchestra.run.smithi135:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f1d75850-7013-11ef-bcea-c7b262605968@mon.b.service' 2024-09-11T08:15:09.447 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T08:15:09.448 INFO:tasks.cephadm.mon.b:Stopped mon.b 2024-09-11T08:15:09.448 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-11T08:15:09.448 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ceph-f1d75850-7013-11ef-bcea-c7b262605968@mgr.y 2024-09-11T08:15:09.733 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:15:09 smithi122 systemd[1]: Stopping Ceph mgr.y for f1d75850-7013-11ef-bcea-c7b262605968... 2024-09-11T08:15:10.091 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:15:09 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[80984]: 2024-09-11T08:15:09.772+0000 7fe238ca6640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.y -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T08:15:10.091 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:15:09 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y[80984]: 2024-09-11T08:15:09.772+0000 7fe238ca6640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-09-11T08:15:10.091 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:15:09 smithi122 podman[82036]: 2024-09-11 08:15:09.8038108 +0000 UTC m=+0.169894418 container died f7fb71753baac7bc3f0456a7610c5143847b6782c6cec3c58c94caf8b86729fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-11T08:15:10.449 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:15:10 smithi122 podman[82036]: 2024-09-11 08:15:10.181320806 +0000 UTC m=+0.547404425 container cleanup f7fb71753baac7bc3f0456a7610c5143847b6782c6cec3c58c94caf8b86729fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True) 2024-09-11T08:15:10.450 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Sep 11 08:15:10 smithi122 bash[82036]: ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-y 2024-09-11T08:15:10.597 DEBUG:teuthology.orchestra.run.smithi122:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f1d75850-7013-11ef-bcea-c7b262605968@mgr.y.service' 2024-09-11T08:15:10.631 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T08:15:10.632 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-11T08:15:10.632 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-11T08:15:10.632 DEBUG:teuthology.orchestra.run.smithi135:> sudo systemctl stop ceph-f1d75850-7013-11ef-bcea-c7b262605968@mgr.x 2024-09-11T08:15:10.961 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:15:10 smithi135 systemd[1]: Stopping Ceph mgr.x for f1d75850-7013-11ef-bcea-c7b262605968... 2024-09-11T08:15:11.216 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:15:10 smithi135 podman[69945]: 2024-09-11 08:15:10.963245641 +0000 UTC m=+0.142765846 container died 4db7f0d5daf8471d85eb45d8f754d389b2724e732ce9ba2c592cc3352a3f6fe2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-11T08:15:11.478 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:15:11 smithi135 podman[69945]: 2024-09-11 08:15:11.298912361 +0000 UTC m=+0.478432566 container cleanup 4db7f0d5daf8471d85eb45d8f754d389b2724e732ce9ba2c592cc3352a3f6fe2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.vendor=CentOS, ceph=True) 2024-09-11T08:15:11.478 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:15:11 smithi135 bash[69945]: ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x 2024-09-11T08:15:11.727 DEBUG:teuthology.orchestra.run.smithi135:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f1d75850-7013-11ef-bcea-c7b262605968@mgr.x.service' 2024-09-11T08:15:11.749 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:15:11 smithi135 podman[69957]: 2024-09-11 08:15:11.49194117 +0000 UTC m=+0.531991569 container remove 4db7f0d5daf8471d85eb45d8f754d389b2724e732ce9ba2c592cc3352a3f6fe2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-mgr-x, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828) 2024-09-11T08:15:11.749 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:15:11 smithi135 systemd[1]: ceph-f1d75850-7013-11ef-bcea-c7b262605968@mgr.x.service: Main process exited, code=exited, status=143/n/a 2024-09-11T08:15:11.749 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:15:11 smithi135 systemd[1]: ceph-f1d75850-7013-11ef-bcea-c7b262605968@mgr.x.service: Failed with result 'exit-code'. 2024-09-11T08:15:11.750 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:15:11 smithi135 systemd[1]: Stopped Ceph mgr.x for f1d75850-7013-11ef-bcea-c7b262605968. 2024-09-11T08:15:11.750 INFO:journalctl@ceph.mgr.x.smithi135.stdout:Sep 11 08:15:11 smithi135 systemd[1]: ceph-f1d75850-7013-11ef-bcea-c7b262605968@mgr.x.service: Consumed 17.459s CPU time. 2024-09-11T08:15:11.762 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T08:15:11.763 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-11T08:15:11.763 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-11T08:15:11.763 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ceph-f1d75850-7013-11ef-bcea-c7b262605968@osd.0 2024-09-11T08:15:12.078 INFO:journalctl@ceph.osd.0.smithi122.stdout:Sep 11 08:15:11 smithi122 systemd[1]: Stopping Ceph osd.0 for f1d75850-7013-11ef-bcea-c7b262605968... 2024-09-11T08:15:12.484 INFO:journalctl@ceph.osd.0.smithi122.stdout:Sep 11 08:15:12 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-0[43596]: 2024-09-11T08:15:12.076+0000 7f969a157640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T08:15:12.484 INFO:journalctl@ceph.osd.0.smithi122.stdout:Sep 11 08:15:12 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-0[43596]: 2024-09-11T08:15:12.076+0000 7f969a157640 -1 osd.0 66 *** Got signal Terminated *** 2024-09-11T08:15:12.484 INFO:journalctl@ceph.osd.0.smithi122.stdout:Sep 11 08:15:12 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-0[43596]: 2024-09-11T08:15:12.076+0000 7f969a157640 -1 osd.0 66 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T08:15:17.887 INFO:journalctl@ceph.osd.0.smithi122.stdout:Sep 11 08:15:17 smithi122 podman[82158]: 2024-09-11 08:15:17.602655789 +0000 UTC m=+5.650238436 container died 7cd3c2182adbf86d3d697a2b2644b03c9e5a37997b015ce499e6e93eb678aa2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-11T08:15:18.220 INFO:journalctl@ceph.osd.0.smithi122.stdout:Sep 11 08:15:17 smithi122 podman[82158]: 2024-09-11 08:15:17.993893718 +0000 UTC m=+6.041476406 container cleanup 7cd3c2182adbf86d3d697a2b2644b03c9e5a37997b015ce499e6e93eb678aa2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828) 2024-09-11T08:15:18.220 INFO:journalctl@ceph.osd.0.smithi122.stdout:Sep 11 08:15:17 smithi122 bash[82158]: ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-0 2024-09-11T08:15:18.483 INFO:journalctl@ceph.osd.0.smithi122.stdout:Sep 11 08:15:18 smithi122 podman[82173]: 2024-09-11 08:15:18.219908143 +0000 UTC m=+0.611963484 container remove 7cd3c2182adbf86d3d697a2b2644b03c9e5a37997b015ce499e6e93eb678aa2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True) 2024-09-11T08:15:18.873 INFO:journalctl@ceph.osd.0.smithi122.stdout:Sep 11 08:15:18 smithi122 podman[82261]: 2024-09-11 08:15:18.429311446 +0000 UTC m=+0.015466073 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T08:15:18.874 INFO:journalctl@ceph.osd.0.smithi122.stdout:Sep 11 08:15:18 smithi122 podman[82261]: 2024-09-11 08:15:18.528932964 +0000 UTC m=+0.115087592 container create e104c20e714175abfcabaeb1ff1fcf591737a5bc60c3548b7db351be7071b260 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-11T08:15:18.874 INFO:journalctl@ceph.osd.0.smithi122.stdout:Sep 11 08:15:18 smithi122 podman[82261]: 2024-09-11 08:15:18.756860373 +0000 UTC m=+0.343015009 container init e104c20e714175abfcabaeb1ff1fcf591737a5bc60c3548b7db351be7071b260 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-11T08:15:18.874 INFO:journalctl@ceph.osd.0.smithi122.stdout:Sep 11 08:15:18 smithi122 podman[82261]: 2024-09-11 08:15:18.760072131 +0000 UTC m=+0.346226764 container start e104c20e714175abfcabaeb1ff1fcf591737a5bc60c3548b7db351be7071b260 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True) 2024-09-11T08:15:18.874 INFO:journalctl@ceph.osd.0.smithi122.stdout:Sep 11 08:15:18 smithi122 podman[82261]: 2024-09-11 08:15:18.873608699 +0000 UTC m=+0.459763345 container attach e104c20e714175abfcabaeb1ff1fcf591737a5bc60c3548b7db351be7071b260 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-0-deactivate, io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-11T08:15:19.233 INFO:journalctl@ceph.osd.0.smithi122.stdout:Sep 11 08:15:18 smithi122 podman[82261]: 2024-09-11 08:15:18.937597541 +0000 UTC m=+0.523752170 container died e104c20e714175abfcabaeb1ff1fcf591737a5bc60c3548b7db351be7071b260 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T08:15:19.543 DEBUG:teuthology.orchestra.run.smithi122:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f1d75850-7013-11ef-bcea-c7b262605968@osd.0.service' 2024-09-11T08:15:19.583 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T08:15:19.583 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-11T08:15:19.583 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-11T08:15:19.583 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ceph-f1d75850-7013-11ef-bcea-c7b262605968@osd.1 2024-09-11T08:15:19.914 INFO:journalctl@ceph.osd.1.smithi122.stdout:Sep 11 08:15:19 smithi122 systemd[1]: Stopping Ceph osd.1 for f1d75850-7013-11ef-bcea-c7b262605968... 2024-09-11T08:15:20.234 INFO:journalctl@ceph.osd.1.smithi122.stdout:Sep 11 08:15:19 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-1[49473]: 2024-09-11T08:15:19.912+0000 7f5871ae5640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T08:15:20.234 INFO:journalctl@ceph.osd.1.smithi122.stdout:Sep 11 08:15:19 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-1[49473]: 2024-09-11T08:15:19.912+0000 7f5871ae5640 -1 osd.1 66 *** Got signal Terminated *** 2024-09-11T08:15:20.234 INFO:journalctl@ceph.osd.1.smithi122.stdout:Sep 11 08:15:19 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-1[49473]: 2024-09-11T08:15:19.912+0000 7f5871ae5640 -1 osd.1 66 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T08:15:25.563 INFO:journalctl@ceph.osd.1.smithi122.stdout:Sep 11 08:15:25 smithi122 podman[82365]: 2024-09-11 08:15:25.289624146 +0000 UTC m=+5.482612091 container died 3af6a71116ec8f831e20bb4dbfda3b745265b66ff2e8e5fded57ea77efa2d68f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-1, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-11T08:15:25.564 INFO:journalctl@ceph.osd.1.smithi122.stdout:Sep 11 08:15:25 smithi122 podman[82365]: 2024-09-11 08:15:25.563195714 +0000 UTC m=+5.756183658 container cleanup 3af6a71116ec8f831e20bb4dbfda3b745265b66ff2e8e5fded57ea77efa2d68f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-1, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828) 2024-09-11T08:15:25.831 INFO:journalctl@ceph.osd.1.smithi122.stdout:Sep 11 08:15:25 smithi122 bash[82365]: ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-1 2024-09-11T08:15:26.184 INFO:journalctl@ceph.osd.1.smithi122.stdout:Sep 11 08:15:25 smithi122 podman[82382]: 2024-09-11 08:15:25.831310618 +0000 UTC m=+0.543447583 container remove 3af6a71116ec8f831e20bb4dbfda3b745265b66ff2e8e5fded57ea77efa2d68f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-1, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T08:15:26.484 INFO:journalctl@ceph.osd.1.smithi122.stdout:Sep 11 08:15:26 smithi122 podman[82478]: 2024-09-11 08:15:26.08686121 +0000 UTC m=+0.015949347 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T08:15:26.484 INFO:journalctl@ceph.osd.1.smithi122.stdout:Sep 11 08:15:26 smithi122 podman[82478]: 2024-09-11 08:15:26.216457329 +0000 UTC m=+0.145545508 container create 6f5b7b33edbe7fca232bca6a63f7dff2adff37f3be65a74897b070c61b971026 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-11T08:15:26.484 INFO:journalctl@ceph.osd.1.smithi122.stdout:Sep 11 08:15:26 smithi122 podman[82478]: 2024-09-11 08:15:26.409252228 +0000 UTC m=+0.338340364 container init 6f5b7b33edbe7fca232bca6a63f7dff2adff37f3be65a74897b070c61b971026 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-11T08:15:26.484 INFO:journalctl@ceph.osd.1.smithi122.stdout:Sep 11 08:15:26 smithi122 podman[82478]: 2024-09-11 08:15:26.412521622 +0000 UTC m=+0.341609762 container start 6f5b7b33edbe7fca232bca6a63f7dff2adff37f3be65a74897b070c61b971026 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-11T08:15:26.879 INFO:journalctl@ceph.osd.1.smithi122.stdout:Sep 11 08:15:26 smithi122 podman[82478]: 2024-09-11 08:15:26.492507115 +0000 UTC m=+0.421595253 container attach 6f5b7b33edbe7fca232bca6a63f7dff2adff37f3be65a74897b070c61b971026 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-11T08:15:26.879 INFO:journalctl@ceph.osd.1.smithi122.stdout:Sep 11 08:15:26 smithi122 podman[82478]: 2024-09-11 08:15:26.589353625 +0000 UTC m=+0.518441763 container died 6f5b7b33edbe7fca232bca6a63f7dff2adff37f3be65a74897b070c61b971026 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, io.buildah.version=1.37.2) 2024-09-11T08:15:27.182 DEBUG:teuthology.orchestra.run.smithi122:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f1d75850-7013-11ef-bcea-c7b262605968@osd.1.service' 2024-09-11T08:15:27.221 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T08:15:27.221 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-11T08:15:27.221 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-11T08:15:27.221 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ceph-f1d75850-7013-11ef-bcea-c7b262605968@osd.2 2024-09-11T08:15:27.484 INFO:journalctl@ceph.osd.2.smithi122.stdout:Sep 11 08:15:27 smithi122 systemd[1]: Stopping Ceph osd.2 for f1d75850-7013-11ef-bcea-c7b262605968... 2024-09-11T08:15:27.984 INFO:journalctl@ceph.osd.2.smithi122.stdout:Sep 11 08:15:27 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-2[55338]: 2024-09-11T08:15:27.606+0000 7efd0bd6d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T08:15:27.984 INFO:journalctl@ceph.osd.2.smithi122.stdout:Sep 11 08:15:27 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-2[55338]: 2024-09-11T08:15:27.606+0000 7efd0bd6d640 -1 osd.2 66 *** Got signal Terminated *** 2024-09-11T08:15:27.984 INFO:journalctl@ceph.osd.2.smithi122.stdout:Sep 11 08:15:27 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-2[55338]: 2024-09-11T08:15:27.606+0000 7efd0bd6d640 -1 osd.2 66 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T08:15:33.234 INFO:journalctl@ceph.osd.2.smithi122.stdout:Sep 11 08:15:32 smithi122 podman[82580]: 2024-09-11 08:15:32.96701267 +0000 UTC m=+5.521079175 container died dc47e4ecc0ba74f90df64e47a05d00814b72950e244457120c92e052e4fc98b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-2, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T08:15:33.733 INFO:journalctl@ceph.osd.2.smithi122.stdout:Sep 11 08:15:33 smithi122 podman[82580]: 2024-09-11 08:15:33.30702106 +0000 UTC m=+5.861087557 container cleanup dc47e4ecc0ba74f90df64e47a05d00814b72950e244457120c92e052e4fc98b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-11T08:15:33.734 INFO:journalctl@ceph.osd.2.smithi122.stdout:Sep 11 08:15:33 smithi122 bash[82580]: ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-2 2024-09-11T08:15:33.734 INFO:journalctl@ceph.osd.2.smithi122.stdout:Sep 11 08:15:33 smithi122 podman[82596]: 2024-09-11 08:15:33.482974595 +0000 UTC m=+0.518355285 container remove dc47e4ecc0ba74f90df64e47a05d00814b72950e244457120c92e052e4fc98b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-11T08:15:34.227 INFO:journalctl@ceph.osd.2.smithi122.stdout:Sep 11 08:15:33 smithi122 podman[82688]: 2024-09-11 08:15:33.820247525 +0000 UTC m=+0.015702044 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T08:15:34.228 INFO:journalctl@ceph.osd.2.smithi122.stdout:Sep 11 08:15:33 smithi122 podman[82688]: 2024-09-11 08:15:33.933837736 +0000 UTC m=+0.129292255 container create e5e234a84b78c26625edb5076190e6737d02153a11d31dd47c1a7c8a07dbcd8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-2-deactivate, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731) 2024-09-11T08:15:34.228 INFO:journalctl@ceph.osd.2.smithi122.stdout:Sep 11 08:15:34 smithi122 podman[82688]: 2024-09-11 08:15:34.144400733 +0000 UTC m=+0.339855256 container init e5e234a84b78c26625edb5076190e6737d02153a11d31dd47c1a7c8a07dbcd8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-2-deactivate, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-11T08:15:34.228 INFO:journalctl@ceph.osd.2.smithi122.stdout:Sep 11 08:15:34 smithi122 podman[82688]: 2024-09-11 08:15:34.147924644 +0000 UTC m=+0.343379164 container start e5e234a84b78c26625edb5076190e6737d02153a11d31dd47c1a7c8a07dbcd8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-2-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True) 2024-09-11T08:15:34.228 INFO:journalctl@ceph.osd.2.smithi122.stdout:Sep 11 08:15:34 smithi122 podman[82688]: 2024-09-11 08:15:34.22755591 +0000 UTC m=+0.423010441 container attach e5e234a84b78c26625edb5076190e6737d02153a11d31dd47c1a7c8a07dbcd8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_BRANCH=HEAD) 2024-09-11T08:15:34.483 INFO:journalctl@ceph.osd.2.smithi122.stdout:Sep 11 08:15:34 smithi122 podman[82688]: 2024-09-11 08:15:34.326290621 +0000 UTC m=+0.521745142 container died e5e234a84b78c26625edb5076190e6737d02153a11d31dd47c1a7c8a07dbcd8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T08:15:34.794 DEBUG:teuthology.orchestra.run.smithi122:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f1d75850-7013-11ef-bcea-c7b262605968@osd.2.service' 2024-09-11T08:15:34.817 INFO:journalctl@ceph.osd.2.smithi122.stdout:Sep 11 08:15:34 smithi122 podman[82711]: 2024-09-11 08:15:34.774443871 +0000 UTC m=+0.437182692 container remove e5e234a84b78c26625edb5076190e6737d02153a11d31dd47c1a7c8a07dbcd8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T08:15:34.817 INFO:journalctl@ceph.osd.2.smithi122.stdout:Sep 11 08:15:34 smithi122 systemd[1]: ceph-f1d75850-7013-11ef-bcea-c7b262605968@osd.2.service: Deactivated successfully. 2024-09-11T08:15:34.818 INFO:journalctl@ceph.osd.2.smithi122.stdout:Sep 11 08:15:34 smithi122 systemd[1]: Stopped Ceph osd.2 for f1d75850-7013-11ef-bcea-c7b262605968. 2024-09-11T08:15:34.818 INFO:journalctl@ceph.osd.2.smithi122.stdout:Sep 11 08:15:34 smithi122 systemd[1]: ceph-f1d75850-7013-11ef-bcea-c7b262605968@osd.2.service: Consumed 5.149s CPU time. 2024-09-11T08:15:34.832 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T08:15:34.832 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-11T08:15:34.833 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-11T08:15:34.833 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ceph-f1d75850-7013-11ef-bcea-c7b262605968@osd.3 2024-09-11T08:15:35.136 INFO:journalctl@ceph.osd.3.smithi122.stdout:Sep 11 08:15:34 smithi122 systemd[1]: Stopping Ceph osd.3 for f1d75850-7013-11ef-bcea-c7b262605968... 2024-09-11T08:15:35.483 INFO:journalctl@ceph.osd.3.smithi122.stdout:Sep 11 08:15:35 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-3[60981]: 2024-09-11T08:15:35.136+0000 7f15cb4b7640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T08:15:35.484 INFO:journalctl@ceph.osd.3.smithi122.stdout:Sep 11 08:15:35 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-3[60981]: 2024-09-11T08:15:35.136+0000 7f15cb4b7640 -1 osd.3 66 *** Got signal Terminated *** 2024-09-11T08:15:35.484 INFO:journalctl@ceph.osd.3.smithi122.stdout:Sep 11 08:15:35 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-3[60981]: 2024-09-11T08:15:35.136+0000 7f15cb4b7640 -1 osd.3 66 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T08:15:38.728 INFO:journalctl@ceph.osd.6.smithi135.stdout:Sep 11 08:15:38 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-6[48960]: 2024-09-11T08:15:38.280+0000 7fa595652640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-11T08:15:15.092751+0000 front 2024-09-11T08:15:15.093071+0000 (oldest deadline 2024-09-11T08:15:37.992793+0000) 2024-09-11T08:15:39.228 INFO:journalctl@ceph.osd.5.smithi135.stdout:Sep 11 08:15:38 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-5[43040]: 2024-09-11T08:15:38.989+0000 7f1769cd5640 -1 osd.5 66 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-11T08:15:16.088429+0000 front 2024-09-11T08:15:16.088468+0000 (oldest deadline 2024-09-11T08:15:38.388298+0000) 2024-09-11T08:15:39.728 INFO:journalctl@ceph.osd.6.smithi135.stdout:Sep 11 08:15:39 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-6[48960]: 2024-09-11T08:15:39.247+0000 7fa595652640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-11T08:15:15.092751+0000 front 2024-09-11T08:15:15.093071+0000 (oldest deadline 2024-09-11T08:15:37.992793+0000) 2024-09-11T08:15:39.996 INFO:journalctl@ceph.osd.7.smithi135.stdout:Sep 11 08:15:39 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-7[54744]: 2024-09-11T08:15:39.852+0000 7faedc045640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-11T08:15:16.734869+0000 front 2024-09-11T08:15:16.734806+0000 (oldest deadline 2024-09-11T08:15:39.034739+0000) 2024-09-11T08:15:40.478 INFO:journalctl@ceph.osd.6.smithi135.stdout:Sep 11 08:15:40 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-6[48960]: 2024-09-11T08:15:40.232+0000 7fa595652640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-11T08:15:15.092751+0000 front 2024-09-11T08:15:15.093071+0000 (oldest deadline 2024-09-11T08:15:37.992793+0000) 2024-09-11T08:15:40.478 INFO:journalctl@ceph.osd.5.smithi135.stdout:Sep 11 08:15:39 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-5[43040]: 2024-09-11T08:15:39.995+0000 7f1769cd5640 -1 osd.5 66 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-11T08:15:16.088429+0000 front 2024-09-11T08:15:16.088468+0000 (oldest deadline 2024-09-11T08:15:38.388298+0000) 2024-09-11T08:15:40.852 INFO:journalctl@ceph.osd.3.smithi122.stdout:Sep 11 08:15:40 smithi122 podman[82789]: 2024-09-11 08:15:40.527029327 +0000 UTC m=+5.469944941 container died 8637de7090a0b16f078e637338a147edd6b35cb22ccc40612951fadf4f9edaca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-3, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-11T08:15:40.852 INFO:journalctl@ceph.osd.3.smithi122.stdout:Sep 11 08:15:40 smithi122 podman[82789]: 2024-09-11 08:15:40.851876026 +0000 UTC m=+5.794791638 container cleanup 8637de7090a0b16f078e637338a147edd6b35cb22ccc40612951fadf4f9edaca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-11T08:15:40.881 INFO:journalctl@ceph.osd.4.smithi135.stdout:Sep 11 08:15:40 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-4[37578]: 2024-09-11T08:15:40.590+0000 7fbc2222c640 -1 osd.4 66 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-11T08:15:15.092648+0000 front 2024-09-11T08:15:15.092741+0000 (oldest deadline 2024-09-11T08:15:39.792689+0000) 2024-09-11T08:15:41.201 INFO:journalctl@ceph.osd.7.smithi135.stdout:Sep 11 08:15:40 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-7[54744]: 2024-09-11T08:15:40.880+0000 7faedc045640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-11T08:15:16.734869+0000 front 2024-09-11T08:15:16.734806+0000 (oldest deadline 2024-09-11T08:15:39.034739+0000) 2024-09-11T08:15:41.202 INFO:journalctl@ceph.osd.5.smithi135.stdout:Sep 11 08:15:41 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-5[43040]: 2024-09-11T08:15:41.012+0000 7f1769cd5640 -1 osd.5 66 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-11T08:15:16.088429+0000 front 2024-09-11T08:15:16.088468+0000 (oldest deadline 2024-09-11T08:15:38.388298+0000) 2024-09-11T08:15:41.233 INFO:journalctl@ceph.osd.3.smithi122.stdout:Sep 11 08:15:40 smithi122 bash[82789]: ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-3 2024-09-11T08:15:41.233 INFO:journalctl@ceph.osd.3.smithi122.stdout:Sep 11 08:15:41 smithi122 podman[82815]: 2024-09-11 08:15:41.019919838 +0000 UTC m=+0.491609025 container remove 8637de7090a0b16f078e637338a147edd6b35cb22ccc40612951fadf4f9edaca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-3, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-11T08:15:41.477 INFO:journalctl@ceph.osd.6.smithi135.stdout:Sep 11 08:15:41 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-6[48960]: 2024-09-11T08:15:41.200+0000 7fa595652640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-11T08:15:15.092751+0000 front 2024-09-11T08:15:15.093071+0000 (oldest deadline 2024-09-11T08:15:37.992793+0000) 2024-09-11T08:15:41.723 INFO:journalctl@ceph.osd.3.smithi122.stdout:Sep 11 08:15:41 smithi122 podman[82906]: 2024-09-11 08:15:41.318470848 +0000 UTC m=+0.015923848 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T08:15:41.723 INFO:journalctl@ceph.osd.3.smithi122.stdout:Sep 11 08:15:41 smithi122 podman[82906]: 2024-09-11 08:15:41.429070493 +0000 UTC m=+0.126523476 container create 41428405391f7fa61d812ae9026ca0c6c428e1abab48d647c025be0e48ee50dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-11T08:15:41.723 INFO:journalctl@ceph.osd.3.smithi122.stdout:Sep 11 08:15:41 smithi122 podman[82906]: 2024-09-11 08:15:41.639670922 +0000 UTC m=+0.337123908 container init 41428405391f7fa61d812ae9026ca0c6c428e1abab48d647c025be0e48ee50dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-3-deactivate, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-11T08:15:41.723 INFO:journalctl@ceph.osd.3.smithi122.stdout:Sep 11 08:15:41 smithi122 podman[82906]: 2024-09-11 08:15:41.643021952 +0000 UTC m=+0.340474935 container start 41428405391f7fa61d812ae9026ca0c6c428e1abab48d647c025be0e48ee50dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD) 2024-09-11T08:15:41.723 INFO:journalctl@ceph.osd.3.smithi122.stdout:Sep 11 08:15:41 smithi122 podman[82906]: 2024-09-11 08:15:41.723005928 +0000 UTC m=+0.420458914 container attach 41428405391f7fa61d812ae9026ca0c6c428e1abab48d647c025be0e48ee50dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-11T08:15:41.893 INFO:journalctl@ceph.osd.4.smithi135.stdout:Sep 11 08:15:41 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-4[37578]: 2024-09-11T08:15:41.590+0000 7fbc2222c640 -1 osd.4 66 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-11T08:15:15.092648+0000 front 2024-09-11T08:15:15.092741+0000 (oldest deadline 2024-09-11T08:15:39.792689+0000) 2024-09-11T08:15:41.984 INFO:journalctl@ceph.osd.3.smithi122.stdout:Sep 11 08:15:41 smithi122 podman[82906]: 2024-09-11 08:15:41.817335621 +0000 UTC m=+0.514788605 container died 41428405391f7fa61d812ae9026ca0c6c428e1abab48d647c025be0e48ee50dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T08:15:42.227 INFO:journalctl@ceph.osd.5.smithi135.stdout:Sep 11 08:15:42 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-5[43040]: 2024-09-11T08:15:42.049+0000 7f1769cd5640 -1 osd.5 66 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-11T08:15:16.088429+0000 front 2024-09-11T08:15:16.088468+0000 (oldest deadline 2024-09-11T08:15:38.388298+0000) 2024-09-11T08:15:42.228 INFO:journalctl@ceph.osd.7.smithi135.stdout:Sep 11 08:15:41 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-7[54744]: 2024-09-11T08:15:41.892+0000 7faedc045640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-11T08:15:16.734869+0000 front 2024-09-11T08:15:16.734806+0000 (oldest deadline 2024-09-11T08:15:39.034739+0000) 2024-09-11T08:15:42.351 DEBUG:teuthology.orchestra.run.smithi122:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f1d75850-7013-11ef-bcea-c7b262605968@osd.3.service' 2024-09-11T08:15:42.373 INFO:journalctl@ceph.osd.3.smithi122.stdout:Sep 11 08:15:42 smithi122 podman[82929]: 2024-09-11 08:15:42.326745712 +0000 UTC m=+0.503568895 container remove 41428405391f7fa61d812ae9026ca0c6c428e1abab48d647c025be0e48ee50dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-11T08:15:42.373 INFO:journalctl@ceph.osd.3.smithi122.stdout:Sep 11 08:15:42 smithi122 systemd[1]: ceph-f1d75850-7013-11ef-bcea-c7b262605968@osd.3.service: Deactivated successfully. 2024-09-11T08:15:42.373 INFO:journalctl@ceph.osd.3.smithi122.stdout:Sep 11 08:15:42 smithi122 systemd[1]: Stopped Ceph osd.3 for f1d75850-7013-11ef-bcea-c7b262605968. 2024-09-11T08:15:42.373 INFO:journalctl@ceph.osd.3.smithi122.stdout:Sep 11 08:15:42 smithi122 systemd[1]: ceph-f1d75850-7013-11ef-bcea-c7b262605968@osd.3.service: Consumed 5.206s CPU time. 2024-09-11T08:15:42.388 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T08:15:42.388 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-11T08:15:42.388 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-11T08:15:42.388 DEBUG:teuthology.orchestra.run.smithi135:> sudo systemctl stop ceph-f1d75850-7013-11ef-bcea-c7b262605968@osd.4 2024-09-11T08:15:42.612 INFO:journalctl@ceph.osd.4.smithi135.stdout:Sep 11 08:15:42 smithi135 systemd[1]: Stopping Ceph osd.4 for f1d75850-7013-11ef-bcea-c7b262605968... 2024-09-11T08:15:42.613 INFO:journalctl@ceph.osd.6.smithi135.stdout:Sep 11 08:15:42 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-6[48960]: 2024-09-11T08:15:42.245+0000 7fa595652640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-11T08:15:15.092751+0000 front 2024-09-11T08:15:15.093071+0000 (oldest deadline 2024-09-11T08:15:37.992793+0000) 2024-09-11T08:15:42.900 INFO:journalctl@ceph.osd.4.smithi135.stdout:Sep 11 08:15:42 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-4[37578]: 2024-09-11T08:15:42.611+0000 7fbc2222c640 -1 osd.4 66 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-11T08:15:15.092648+0000 front 2024-09-11T08:15:15.092741+0000 (oldest deadline 2024-09-11T08:15:39.792689+0000) 2024-09-11T08:15:42.900 INFO:journalctl@ceph.osd.4.smithi135.stdout:Sep 11 08:15:42 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-4[37578]: 2024-09-11T08:15:42.692+0000 7fbc24411640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T08:15:42.900 INFO:journalctl@ceph.osd.4.smithi135.stdout:Sep 11 08:15:42 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-4[37578]: 2024-09-11T08:15:42.692+0000 7fbc24411640 -1 osd.4 66 *** Got signal Terminated *** 2024-09-11T08:15:42.900 INFO:journalctl@ceph.osd.4.smithi135.stdout:Sep 11 08:15:42 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-4[37578]: 2024-09-11T08:15:42.692+0000 7fbc24411640 -1 osd.4 66 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T08:15:43.228 INFO:journalctl@ceph.osd.7.smithi135.stdout:Sep 11 08:15:42 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-7[54744]: 2024-09-11T08:15:42.899+0000 7faedc045640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-11T08:15:16.734869+0000 front 2024-09-11T08:15:16.734806+0000 (oldest deadline 2024-09-11T08:15:39.034739+0000) 2024-09-11T08:15:43.228 INFO:journalctl@ceph.osd.5.smithi135.stdout:Sep 11 08:15:43 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-5[43040]: 2024-09-11T08:15:43.056+0000 7f1769cd5640 -1 osd.5 66 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-11T08:15:16.088429+0000 front 2024-09-11T08:15:16.088468+0000 (oldest deadline 2024-09-11T08:15:38.388298+0000) 2024-09-11T08:15:43.644 INFO:journalctl@ceph.osd.6.smithi135.stdout:Sep 11 08:15:43 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-6[48960]: 2024-09-11T08:15:43.272+0000 7fa595652640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-11T08:15:15.092751+0000 front 2024-09-11T08:15:15.093071+0000 (oldest deadline 2024-09-11T08:15:37.992793+0000) 2024-09-11T08:15:43.945 INFO:journalctl@ceph.osd.4.smithi135.stdout:Sep 11 08:15:43 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-4[37578]: 2024-09-11T08:15:43.644+0000 7fbc2222c640 -1 osd.4 66 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-11T08:15:15.092648+0000 front 2024-09-11T08:15:15.092741+0000 (oldest deadline 2024-09-11T08:15:39.792689+0000) 2024-09-11T08:15:44.228 INFO:journalctl@ceph.osd.7.smithi135.stdout:Sep 11 08:15:43 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-7[54744]: 2024-09-11T08:15:43.945+0000 7faedc045640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-11T08:15:16.734869+0000 front 2024-09-11T08:15:16.734806+0000 (oldest deadline 2024-09-11T08:15:39.034739+0000) 2024-09-11T08:15:44.228 INFO:journalctl@ceph.osd.5.smithi135.stdout:Sep 11 08:15:44 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-5[43040]: 2024-09-11T08:15:44.071+0000 7f1769cd5640 -1 osd.5 66 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-11T08:15:16.088429+0000 front 2024-09-11T08:15:16.088468+0000 (oldest deadline 2024-09-11T08:15:38.388298+0000) 2024-09-11T08:15:44.653 INFO:journalctl@ceph.osd.6.smithi135.stdout:Sep 11 08:15:44 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-6[48960]: 2024-09-11T08:15:44.312+0000 7fa595652640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-11T08:15:15.092751+0000 front 2024-09-11T08:15:15.093071+0000 (oldest deadline 2024-09-11T08:15:37.992793+0000) 2024-09-11T08:15:44.978 INFO:journalctl@ceph.osd.4.smithi135.stdout:Sep 11 08:15:44 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-4[37578]: 2024-09-11T08:15:44.652+0000 7fbc2222c640 -1 osd.4 66 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-11T08:15:15.092648+0000 front 2024-09-11T08:15:15.092741+0000 (oldest deadline 2024-09-11T08:15:39.792689+0000) 2024-09-11T08:15:44.978 INFO:journalctl@ceph.osd.7.smithi135.stdout:Sep 11 08:15:44 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-7[54744]: 2024-09-11T08:15:44.902+0000 7faedc045640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-11T08:15:16.734869+0000 front 2024-09-11T08:15:16.734806+0000 (oldest deadline 2024-09-11T08:15:39.034739+0000) 2024-09-11T08:15:45.478 INFO:journalctl@ceph.osd.5.smithi135.stdout:Sep 11 08:15:45 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-5[43040]: 2024-09-11T08:15:45.109+0000 7f1769cd5640 -1 osd.5 66 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-11T08:15:16.088429+0000 front 2024-09-11T08:15:16.088468+0000 (oldest deadline 2024-09-11T08:15:38.388298+0000) 2024-09-11T08:15:45.478 INFO:journalctl@ceph.osd.6.smithi135.stdout:Sep 11 08:15:45 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-6[48960]: 2024-09-11T08:15:45.347+0000 7fa595652640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-11T08:15:15.092751+0000 front 2024-09-11T08:15:15.093071+0000 (oldest deadline 2024-09-11T08:15:37.992793+0000) 2024-09-11T08:15:45.978 INFO:journalctl@ceph.osd.4.smithi135.stdout:Sep 11 08:15:45 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-4[37578]: 2024-09-11T08:15:45.693+0000 7fbc2222c640 -1 osd.4 66 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-11T08:15:15.092648+0000 front 2024-09-11T08:15:15.092741+0000 (oldest deadline 2024-09-11T08:15:39.792689+0000) 2024-09-11T08:15:45.978 INFO:journalctl@ceph.osd.7.smithi135.stdout:Sep 11 08:15:45 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-7[54744]: 2024-09-11T08:15:45.913+0000 7faedc045640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-11T08:15:16.734869+0000 front 2024-09-11T08:15:16.734806+0000 (oldest deadline 2024-09-11T08:15:39.034739+0000) 2024-09-11T08:15:46.387 INFO:journalctl@ceph.osd.5.smithi135.stdout:Sep 11 08:15:46 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-5[43040]: 2024-09-11T08:15:46.071+0000 7f1769cd5640 -1 osd.5 66 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-11T08:15:16.088429+0000 front 2024-09-11T08:15:16.088468+0000 (oldest deadline 2024-09-11T08:15:38.388298+0000) 2024-09-11T08:15:46.706 INFO:journalctl@ceph.osd.6.smithi135.stdout:Sep 11 08:15:46 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-6[48960]: 2024-09-11T08:15:46.387+0000 7fa595652640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-11T08:15:15.092751+0000 front 2024-09-11T08:15:15.093071+0000 (oldest deadline 2024-09-11T08:15:37.992793+0000) 2024-09-11T08:15:46.706 INFO:journalctl@ceph.osd.6.smithi135.stdout:Sep 11 08:15:46 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-6[48960]: 2024-09-11T08:15:46.387+0000 7fa595652640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.122:6814 osd.1 since back 2024-09-11T08:15:23.293653+0000 front 2024-09-11T08:15:23.293712+0000 (oldest deadline 2024-09-11T08:15:46.093854+0000) 2024-09-11T08:15:46.978 INFO:journalctl@ceph.osd.7.smithi135.stdout:Sep 11 08:15:46 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-7[54744]: 2024-09-11T08:15:46.927+0000 7faedc045640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-11T08:15:16.734869+0000 front 2024-09-11T08:15:16.734806+0000 (oldest deadline 2024-09-11T08:15:39.034739+0000) 2024-09-11T08:15:46.978 INFO:journalctl@ceph.osd.4.smithi135.stdout:Sep 11 08:15:46 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-4[37578]: 2024-09-11T08:15:46.706+0000 7fbc2222c640 -1 osd.4 66 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-11T08:15:15.092648+0000 front 2024-09-11T08:15:15.092741+0000 (oldest deadline 2024-09-11T08:15:39.792689+0000) 2024-09-11T08:15:47.413 INFO:journalctl@ceph.osd.5.smithi135.stdout:Sep 11 08:15:47 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-5[43040]: 2024-09-11T08:15:47.056+0000 7f1769cd5640 -1 osd.5 66 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-11T08:15:16.088429+0000 front 2024-09-11T08:15:16.088468+0000 (oldest deadline 2024-09-11T08:15:38.388298+0000) 2024-09-11T08:15:47.728 INFO:journalctl@ceph.osd.6.smithi135.stdout:Sep 11 08:15:47 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-6[48960]: 2024-09-11T08:15:47.413+0000 7fa595652640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-11T08:15:15.092751+0000 front 2024-09-11T08:15:15.093071+0000 (oldest deadline 2024-09-11T08:15:37.992793+0000) 2024-09-11T08:15:47.728 INFO:journalctl@ceph.osd.6.smithi135.stdout:Sep 11 08:15:47 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-6[48960]: 2024-09-11T08:15:47.413+0000 7fa595652640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.122:6814 osd.1 since back 2024-09-11T08:15:23.293653+0000 front 2024-09-11T08:15:23.293712+0000 (oldest deadline 2024-09-11T08:15:46.093854+0000) 2024-09-11T08:15:48.228 INFO:journalctl@ceph.osd.4.smithi135.stdout:Sep 11 08:15:48 smithi135 podman[70082]: 2024-09-11 08:15:48.04985468 +0000 UTC m=+5.459801934 container died 55e098de61bca8f8c0d2b6fc8bc76197b191fd89daac9523fca299aa3bd0185a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-4, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-11T08:15:48.228 INFO:journalctl@ceph.osd.7.smithi135.stdout:Sep 11 08:15:47 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-7[54744]: 2024-09-11T08:15:47.956+0000 7faedc045640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-11T08:15:16.734869+0000 front 2024-09-11T08:15:16.734806+0000 (oldest deadline 2024-09-11T08:15:39.034739+0000) 2024-09-11T08:15:48.228 INFO:journalctl@ceph.osd.7.smithi135.stdout:Sep 11 08:15:47 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-7[54744]: 2024-09-11T08:15:47.956+0000 7faedc045640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.122:6814 osd.1 since back 2024-09-11T08:15:19.035249+0000 front 2024-09-11T08:15:19.035235+0000 (oldest deadline 2024-09-11T08:15:47.835608+0000) 2024-09-11T08:15:48.228 INFO:journalctl@ceph.osd.5.smithi135.stdout:Sep 11 08:15:48 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-5[43040]: 2024-09-11T08:15:48.015+0000 7f1769cd5640 -1 osd.5 66 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-11T08:15:16.088429+0000 front 2024-09-11T08:15:16.088468+0000 (oldest deadline 2024-09-11T08:15:38.388298+0000) 2024-09-11T08:15:48.677 INFO:journalctl@ceph.osd.4.smithi135.stdout:Sep 11 08:15:48 smithi135 podman[70082]: 2024-09-11 08:15:48.392198085 +0000 UTC m=+5.802145329 container cleanup 55e098de61bca8f8c0d2b6fc8bc76197b191fd89daac9523fca299aa3bd0185a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-11T08:15:48.677 INFO:journalctl@ceph.osd.4.smithi135.stdout:Sep 11 08:15:48 smithi135 bash[70082]: ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-4 2024-09-11T08:15:48.677 INFO:journalctl@ceph.osd.6.smithi135.stdout:Sep 11 08:15:48 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-6[48960]: 2024-09-11T08:15:48.406+0000 7fa595652640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-11T08:15:15.092751+0000 front 2024-09-11T08:15:15.093071+0000 (oldest deadline 2024-09-11T08:15:37.992793+0000) 2024-09-11T08:15:48.677 INFO:journalctl@ceph.osd.6.smithi135.stdout:Sep 11 08:15:48 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-6[48960]: 2024-09-11T08:15:48.406+0000 7fa595652640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.122:6814 osd.1 since back 2024-09-11T08:15:23.293653+0000 front 2024-09-11T08:15:23.293712+0000 (oldest deadline 2024-09-11T08:15:46.093854+0000) 2024-09-11T08:15:48.956 INFO:journalctl@ceph.osd.4.smithi135.stdout:Sep 11 08:15:48 smithi135 podman[70097]: 2024-09-11 08:15:48.676661853 +0000 UTC m=+0.627979949 container remove 55e098de61bca8f8c0d2b6fc8bc76197b191fd89daac9523fca299aa3bd0185a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-4, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2) 2024-09-11T08:15:49.228 INFO:journalctl@ceph.osd.4.smithi135.stdout:Sep 11 08:15:49 smithi135 podman[70186]: 2024-09-11 08:15:48.914505852 +0000 UTC m=+0.015853310 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T08:15:49.228 INFO:journalctl@ceph.osd.4.smithi135.stdout:Sep 11 08:15:49 smithi135 podman[70186]: 2024-09-11 08:15:49.019130843 +0000 UTC m=+0.120478302 container create fbb9c3c10d62c01dc4d688bfaed08fad9aa4910a6556d6e24e211ff50d02283f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-11T08:15:49.229 INFO:journalctl@ceph.osd.5.smithi135.stdout:Sep 11 08:15:49 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-5[43040]: 2024-09-11T08:15:49.016+0000 7f1769cd5640 -1 osd.5 66 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-11T08:15:16.088429+0000 front 2024-09-11T08:15:16.088468+0000 (oldest deadline 2024-09-11T08:15:38.388298+0000) 2024-09-11T08:15:49.229 INFO:journalctl@ceph.osd.7.smithi135.stdout:Sep 11 08:15:48 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-7[54744]: 2024-09-11T08:15:48.955+0000 7faedc045640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-11T08:15:16.734869+0000 front 2024-09-11T08:15:16.734806+0000 (oldest deadline 2024-09-11T08:15:39.034739+0000) 2024-09-11T08:15:49.229 INFO:journalctl@ceph.osd.7.smithi135.stdout:Sep 11 08:15:48 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-7[54744]: 2024-09-11T08:15:48.955+0000 7faedc045640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.122:6814 osd.1 since back 2024-09-11T08:15:19.035249+0000 front 2024-09-11T08:15:19.035235+0000 (oldest deadline 2024-09-11T08:15:47.835608+0000) 2024-09-11T08:15:49.692 INFO:journalctl@ceph.osd.6.smithi135.stdout:Sep 11 08:15:49 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-6[48960]: 2024-09-11T08:15:49.431+0000 7fa595652640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-11T08:15:15.092751+0000 front 2024-09-11T08:15:15.093071+0000 (oldest deadline 2024-09-11T08:15:37.992793+0000) 2024-09-11T08:15:49.692 INFO:journalctl@ceph.osd.6.smithi135.stdout:Sep 11 08:15:49 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-6[48960]: 2024-09-11T08:15:49.431+0000 7fa595652640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.122:6814 osd.1 since back 2024-09-11T08:15:23.293653+0000 front 2024-09-11T08:15:23.293712+0000 (oldest deadline 2024-09-11T08:15:46.093854+0000) 2024-09-11T08:15:49.693 INFO:journalctl@ceph.osd.4.smithi135.stdout:Sep 11 08:15:49 smithi135 podman[70186]: 2024-09-11 08:15:49.23960142 +0000 UTC m=+0.340948875 container init fbb9c3c10d62c01dc4d688bfaed08fad9aa4910a6556d6e24e211ff50d02283f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-4-deactivate, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-11T08:15:49.693 INFO:journalctl@ceph.osd.4.smithi135.stdout:Sep 11 08:15:49 smithi135 podman[70186]: 2024-09-11 08:15:49.24293561 +0000 UTC m=+0.344283065 container start fbb9c3c10d62c01dc4d688bfaed08fad9aa4910a6556d6e24e211ff50d02283f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True) 2024-09-11T08:15:49.693 INFO:journalctl@ceph.osd.4.smithi135.stdout:Sep 11 08:15:49 smithi135 podman[70186]: 2024-09-11 08:15:49.338441946 +0000 UTC m=+0.439789421 container attach fbb9c3c10d62c01dc4d688bfaed08fad9aa4910a6556d6e24e211ff50d02283f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_CLEAN=True) 2024-09-11T08:15:49.693 INFO:journalctl@ceph.osd.4.smithi135.stdout:Sep 11 08:15:49 smithi135 podman[70186]: 2024-09-11 08:15:49.421507758 +0000 UTC m=+0.522855220 container died fbb9c3c10d62c01dc4d688bfaed08fad9aa4910a6556d6e24e211ff50d02283f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-4-deactivate, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-11T08:15:50.005 DEBUG:teuthology.orchestra.run.smithi135:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f1d75850-7013-11ef-bcea-c7b262605968@osd.4.service' 2024-09-11T08:15:50.044 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T08:15:50.044 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-11T08:15:50.045 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-11T08:15:50.045 DEBUG:teuthology.orchestra.run.smithi135:> sudo systemctl stop ceph-f1d75850-7013-11ef-bcea-c7b262605968@osd.5 2024-09-11T08:15:50.228 INFO:journalctl@ceph.osd.7.smithi135.stdout:Sep 11 08:15:49 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-7[54744]: 2024-09-11T08:15:49.974+0000 7faedc045640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-11T08:15:16.734869+0000 front 2024-09-11T08:15:16.734806+0000 (oldest deadline 2024-09-11T08:15:39.034739+0000) 2024-09-11T08:15:50.228 INFO:journalctl@ceph.osd.7.smithi135.stdout:Sep 11 08:15:49 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-7[54744]: 2024-09-11T08:15:49.974+0000 7faedc045640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.122:6814 osd.1 since back 2024-09-11T08:15:19.035249+0000 front 2024-09-11T08:15:19.035235+0000 (oldest deadline 2024-09-11T08:15:47.835608+0000) 2024-09-11T08:15:50.228 INFO:journalctl@ceph.osd.5.smithi135.stdout:Sep 11 08:15:50 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-5[43040]: 2024-09-11T08:15:50.033+0000 7f1769cd5640 -1 osd.5 66 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-11T08:15:16.088429+0000 front 2024-09-11T08:15:16.088468+0000 (oldest deadline 2024-09-11T08:15:38.388298+0000) 2024-09-11T08:15:50.228 INFO:journalctl@ceph.osd.5.smithi135.stdout:Sep 11 08:15:50 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-5[43040]: 2024-09-11T08:15:50.033+0000 7f1769cd5640 -1 osd.5 66 heartbeat_check: no reply from 172.21.15.122:6814 osd.1 since back 2024-09-11T08:15:23.689254+0000 front 2024-09-11T08:15:23.689125+0000 (oldest deadline 2024-09-11T08:15:49.589211+0000) 2024-09-11T08:15:50.228 INFO:journalctl@ceph.osd.5.smithi135.stdout:Sep 11 08:15:50 smithi135 systemd[1]: Stopping Ceph osd.5 for f1d75850-7013-11ef-bcea-c7b262605968... 2024-09-11T08:15:50.728 INFO:journalctl@ceph.osd.5.smithi135.stdout:Sep 11 08:15:50 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-5[43040]: 2024-09-11T08:15:50.403+0000 7f176c6bb640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T08:15:50.728 INFO:journalctl@ceph.osd.5.smithi135.stdout:Sep 11 08:15:50 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-5[43040]: 2024-09-11T08:15:50.403+0000 7f176c6bb640 -1 osd.5 66 *** Got signal Terminated *** 2024-09-11T08:15:50.728 INFO:journalctl@ceph.osd.5.smithi135.stdout:Sep 11 08:15:50 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-5[43040]: 2024-09-11T08:15:50.403+0000 7f176c6bb640 -1 osd.5 66 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T08:15:50.728 INFO:journalctl@ceph.osd.6.smithi135.stdout:Sep 11 08:15:50 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-6[48960]: 2024-09-11T08:15:50.434+0000 7fa595652640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-11T08:15:15.092751+0000 front 2024-09-11T08:15:15.093071+0000 (oldest deadline 2024-09-11T08:15:37.992793+0000) 2024-09-11T08:15:50.728 INFO:journalctl@ceph.osd.6.smithi135.stdout:Sep 11 08:15:50 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-6[48960]: 2024-09-11T08:15:50.434+0000 7fa595652640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.122:6814 osd.1 since back 2024-09-11T08:15:23.293653+0000 front 2024-09-11T08:15:23.293712+0000 (oldest deadline 2024-09-11T08:15:46.093854+0000) 2024-09-11T08:15:51.437 INFO:journalctl@ceph.osd.7.smithi135.stdout:Sep 11 08:15:50 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-7[54744]: 2024-09-11T08:15:50.998+0000 7faedc045640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-11T08:15:16.734869+0000 front 2024-09-11T08:15:16.734806+0000 (oldest deadline 2024-09-11T08:15:39.034739+0000) 2024-09-11T08:15:51.437 INFO:journalctl@ceph.osd.7.smithi135.stdout:Sep 11 08:15:50 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-7[54744]: 2024-09-11T08:15:50.998+0000 7faedc045640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.122:6814 osd.1 since back 2024-09-11T08:15:19.035249+0000 front 2024-09-11T08:15:19.035235+0000 (oldest deadline 2024-09-11T08:15:47.835608+0000) 2024-09-11T08:15:51.437 INFO:journalctl@ceph.osd.5.smithi135.stdout:Sep 11 08:15:51 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-5[43040]: 2024-09-11T08:15:51.023+0000 7f1769cd5640 -1 osd.5 66 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-11T08:15:16.088429+0000 front 2024-09-11T08:15:16.088468+0000 (oldest deadline 2024-09-11T08:15:38.388298+0000) 2024-09-11T08:15:51.437 INFO:journalctl@ceph.osd.5.smithi135.stdout:Sep 11 08:15:51 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-5[43040]: 2024-09-11T08:15:51.023+0000 7f1769cd5640 -1 osd.5 66 heartbeat_check: no reply from 172.21.15.122:6814 osd.1 since back 2024-09-11T08:15:23.689254+0000 front 2024-09-11T08:15:23.689125+0000 (oldest deadline 2024-09-11T08:15:49.589211+0000) 2024-09-11T08:15:51.727 INFO:journalctl@ceph.osd.6.smithi135.stdout:Sep 11 08:15:51 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-6[48960]: 2024-09-11T08:15:51.435+0000 7fa595652640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-11T08:15:15.092751+0000 front 2024-09-11T08:15:15.093071+0000 (oldest deadline 2024-09-11T08:15:37.992793+0000) 2024-09-11T08:15:51.728 INFO:journalctl@ceph.osd.6.smithi135.stdout:Sep 11 08:15:51 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-6[48960]: 2024-09-11T08:15:51.435+0000 7fa595652640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.122:6814 osd.1 since back 2024-09-11T08:15:23.293653+0000 front 2024-09-11T08:15:23.293712+0000 (oldest deadline 2024-09-11T08:15:46.093854+0000) 2024-09-11T08:15:52.463 INFO:journalctl@ceph.osd.5.smithi135.stdout:Sep 11 08:15:51 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-5[43040]: 2024-09-11T08:15:51.981+0000 7f1769cd5640 -1 osd.5 66 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-11T08:15:16.088429+0000 front 2024-09-11T08:15:16.088468+0000 (oldest deadline 2024-09-11T08:15:38.388298+0000) 2024-09-11T08:15:52.463 INFO:journalctl@ceph.osd.5.smithi135.stdout:Sep 11 08:15:51 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-5[43040]: 2024-09-11T08:15:51.981+0000 7f1769cd5640 -1 osd.5 66 heartbeat_check: no reply from 172.21.15.122:6814 osd.1 since back 2024-09-11T08:15:23.689254+0000 front 2024-09-11T08:15:23.689125+0000 (oldest deadline 2024-09-11T08:15:49.589211+0000) 2024-09-11T08:15:52.463 INFO:journalctl@ceph.osd.7.smithi135.stdout:Sep 11 08:15:52 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-7[54744]: 2024-09-11T08:15:52.033+0000 7faedc045640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-11T08:15:16.734869+0000 front 2024-09-11T08:15:16.734806+0000 (oldest deadline 2024-09-11T08:15:39.034739+0000) 2024-09-11T08:15:52.463 INFO:journalctl@ceph.osd.7.smithi135.stdout:Sep 11 08:15:52 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-7[54744]: 2024-09-11T08:15:52.033+0000 7faedc045640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.122:6814 osd.1 since back 2024-09-11T08:15:19.035249+0000 front 2024-09-11T08:15:19.035235+0000 (oldest deadline 2024-09-11T08:15:47.835608+0000) 2024-09-11T08:15:52.727 INFO:journalctl@ceph.osd.6.smithi135.stdout:Sep 11 08:15:52 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-6[48960]: 2024-09-11T08:15:52.461+0000 7fa595652640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-11T08:15:15.092751+0000 front 2024-09-11T08:15:15.093071+0000 (oldest deadline 2024-09-11T08:15:37.992793+0000) 2024-09-11T08:15:52.728 INFO:journalctl@ceph.osd.6.smithi135.stdout:Sep 11 08:15:52 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-6[48960]: 2024-09-11T08:15:52.461+0000 7fa595652640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.122:6814 osd.1 since back 2024-09-11T08:15:23.293653+0000 front 2024-09-11T08:15:23.293712+0000 (oldest deadline 2024-09-11T08:15:46.093854+0000) 2024-09-11T08:15:53.478 INFO:journalctl@ceph.osd.5.smithi135.stdout:Sep 11 08:15:53 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-5[43040]: 2024-09-11T08:15:53.028+0000 7f1769cd5640 -1 osd.5 66 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-11T08:15:16.088429+0000 front 2024-09-11T08:15:16.088468+0000 (oldest deadline 2024-09-11T08:15:38.388298+0000) 2024-09-11T08:15:53.478 INFO:journalctl@ceph.osd.5.smithi135.stdout:Sep 11 08:15:53 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-5[43040]: 2024-09-11T08:15:53.028+0000 7f1769cd5640 -1 osd.5 66 heartbeat_check: no reply from 172.21.15.122:6814 osd.1 since back 2024-09-11T08:15:23.689254+0000 front 2024-09-11T08:15:23.689125+0000 (oldest deadline 2024-09-11T08:15:49.589211+0000) 2024-09-11T08:15:53.478 INFO:journalctl@ceph.osd.7.smithi135.stdout:Sep 11 08:15:53 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-7[54744]: 2024-09-11T08:15:53.055+0000 7faedc045640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-11T08:15:16.734869+0000 front 2024-09-11T08:15:16.734806+0000 (oldest deadline 2024-09-11T08:15:39.034739+0000) 2024-09-11T08:15:53.478 INFO:journalctl@ceph.osd.7.smithi135.stdout:Sep 11 08:15:53 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-7[54744]: 2024-09-11T08:15:53.055+0000 7faedc045640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.122:6814 osd.1 since back 2024-09-11T08:15:19.035249+0000 front 2024-09-11T08:15:19.035235+0000 (oldest deadline 2024-09-11T08:15:47.835608+0000) 2024-09-11T08:15:53.478 INFO:journalctl@ceph.osd.7.smithi135.stdout:Sep 11 08:15:53 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-7[54744]: 2024-09-11T08:15:53.055+0000 7faedc045640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.122:6822 osd.2 since back 2024-09-11T08:15:29.536654+0000 front 2024-09-11T08:15:29.536686+0000 (oldest deadline 2024-09-11T08:15:53.036521+0000) 2024-09-11T08:15:53.978 INFO:journalctl@ceph.osd.6.smithi135.stdout:Sep 11 08:15:53 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-6[48960]: 2024-09-11T08:15:53.493+0000 7fa595652640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-11T08:15:15.092751+0000 front 2024-09-11T08:15:15.093071+0000 (oldest deadline 2024-09-11T08:15:37.992793+0000) 2024-09-11T08:15:53.978 INFO:journalctl@ceph.osd.6.smithi135.stdout:Sep 11 08:15:53 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-6[48960]: 2024-09-11T08:15:53.493+0000 7fa595652640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.122:6814 osd.1 since back 2024-09-11T08:15:23.293653+0000 front 2024-09-11T08:15:23.293712+0000 (oldest deadline 2024-09-11T08:15:46.093854+0000) 2024-09-11T08:15:54.478 INFO:journalctl@ceph.osd.5.smithi135.stdout:Sep 11 08:15:54 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-5[43040]: 2024-09-11T08:15:54.059+0000 7f1769cd5640 -1 osd.5 66 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-11T08:15:16.088429+0000 front 2024-09-11T08:15:16.088468+0000 (oldest deadline 2024-09-11T08:15:38.388298+0000) 2024-09-11T08:15:54.478 INFO:journalctl@ceph.osd.5.smithi135.stdout:Sep 11 08:15:54 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-5[43040]: 2024-09-11T08:15:54.059+0000 7f1769cd5640 -1 osd.5 66 heartbeat_check: no reply from 172.21.15.122:6814 osd.1 since back 2024-09-11T08:15:23.689254+0000 front 2024-09-11T08:15:23.689125+0000 (oldest deadline 2024-09-11T08:15:49.589211+0000) 2024-09-11T08:15:54.478 INFO:journalctl@ceph.osd.7.smithi135.stdout:Sep 11 08:15:54 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-7[54744]: 2024-09-11T08:15:54.036+0000 7faedc045640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-11T08:15:16.734869+0000 front 2024-09-11T08:15:16.734806+0000 (oldest deadline 2024-09-11T08:15:39.034739+0000) 2024-09-11T08:15:54.478 INFO:journalctl@ceph.osd.7.smithi135.stdout:Sep 11 08:15:54 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-7[54744]: 2024-09-11T08:15:54.036+0000 7faedc045640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.122:6814 osd.1 since back 2024-09-11T08:15:19.035249+0000 front 2024-09-11T08:15:19.035235+0000 (oldest deadline 2024-09-11T08:15:47.835608+0000) 2024-09-11T08:15:54.478 INFO:journalctl@ceph.osd.7.smithi135.stdout:Sep 11 08:15:54 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-7[54744]: 2024-09-11T08:15:54.036+0000 7faedc045640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.122:6822 osd.2 since back 2024-09-11T08:15:29.536654+0000 front 2024-09-11T08:15:29.536686+0000 (oldest deadline 2024-09-11T08:15:53.036521+0000) 2024-09-11T08:15:54.978 INFO:journalctl@ceph.osd.6.smithi135.stdout:Sep 11 08:15:54 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-6[48960]: 2024-09-11T08:15:54.539+0000 7fa595652640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-11T08:15:15.092751+0000 front 2024-09-11T08:15:15.093071+0000 (oldest deadline 2024-09-11T08:15:37.992793+0000) 2024-09-11T08:15:54.978 INFO:journalctl@ceph.osd.6.smithi135.stdout:Sep 11 08:15:54 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-6[48960]: 2024-09-11T08:15:54.539+0000 7fa595652640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.122:6814 osd.1 since back 2024-09-11T08:15:23.293653+0000 front 2024-09-11T08:15:23.293712+0000 (oldest deadline 2024-09-11T08:15:46.093854+0000) 2024-09-11T08:15:55.478 INFO:journalctl@ceph.osd.5.smithi135.stdout:Sep 11 08:15:55 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-5[43040]: 2024-09-11T08:15:55.035+0000 7f1769cd5640 -1 osd.5 66 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-11T08:15:16.088429+0000 front 2024-09-11T08:15:16.088468+0000 (oldest deadline 2024-09-11T08:15:38.388298+0000) 2024-09-11T08:15:55.479 INFO:journalctl@ceph.osd.5.smithi135.stdout:Sep 11 08:15:55 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-5[43040]: 2024-09-11T08:15:55.035+0000 7f1769cd5640 -1 osd.5 66 heartbeat_check: no reply from 172.21.15.122:6814 osd.1 since back 2024-09-11T08:15:23.689254+0000 front 2024-09-11T08:15:23.689125+0000 (oldest deadline 2024-09-11T08:15:49.589211+0000) 2024-09-11T08:15:55.479 INFO:journalctl@ceph.osd.7.smithi135.stdout:Sep 11 08:15:55 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-7[54744]: 2024-09-11T08:15:55.073+0000 7faedc045640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-11T08:15:16.734869+0000 front 2024-09-11T08:15:16.734806+0000 (oldest deadline 2024-09-11T08:15:39.034739+0000) 2024-09-11T08:15:55.479 INFO:journalctl@ceph.osd.7.smithi135.stdout:Sep 11 08:15:55 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-7[54744]: 2024-09-11T08:15:55.073+0000 7faedc045640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.122:6814 osd.1 since back 2024-09-11T08:15:19.035249+0000 front 2024-09-11T08:15:19.035235+0000 (oldest deadline 2024-09-11T08:15:47.835608+0000) 2024-09-11T08:15:55.479 INFO:journalctl@ceph.osd.7.smithi135.stdout:Sep 11 08:15:55 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-7[54744]: 2024-09-11T08:15:55.073+0000 7faedc045640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.122:6822 osd.2 since back 2024-09-11T08:15:29.536654+0000 front 2024-09-11T08:15:29.536686+0000 (oldest deadline 2024-09-11T08:15:53.036521+0000) 2024-09-11T08:15:55.853 INFO:journalctl@ceph.osd.6.smithi135.stdout:Sep 11 08:15:55 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-6[48960]: 2024-09-11T08:15:55.495+0000 7fa595652640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-11T08:15:15.092751+0000 front 2024-09-11T08:15:15.093071+0000 (oldest deadline 2024-09-11T08:15:37.992793+0000) 2024-09-11T08:15:55.853 INFO:journalctl@ceph.osd.6.smithi135.stdout:Sep 11 08:15:55 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-6[48960]: 2024-09-11T08:15:55.495+0000 7fa595652640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.122:6814 osd.1 since back 2024-09-11T08:15:23.293653+0000 front 2024-09-11T08:15:23.293712+0000 (oldest deadline 2024-09-11T08:15:46.093854+0000) 2024-09-11T08:15:55.853 INFO:journalctl@ceph.osd.6.smithi135.stdout:Sep 11 08:15:55 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-6[48960]: 2024-09-11T08:15:55.495+0000 7fa595652640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.122:6822 osd.2 since back 2024-09-11T08:15:30.695537+0000 front 2024-09-11T08:15:30.695760+0000 (oldest deadline 2024-09-11T08:15:54.795367+0000) 2024-09-11T08:15:56.171 INFO:journalctl@ceph.osd.5.smithi135.stdout:Sep 11 08:15:55 smithi135 podman[70288]: 2024-09-11 08:15:55.852874779 +0000 UTC m=+5.571120257 container died d6e2740c975c19c040c381535a73534ee6725c60b01bfa82659f923faeeb2ae0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-11T08:15:56.171 INFO:journalctl@ceph.osd.7.smithi135.stdout:Sep 11 08:15:56 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-7[54744]: 2024-09-11T08:15:56.025+0000 7faedc045640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-11T08:15:16.734869+0000 front 2024-09-11T08:15:16.734806+0000 (oldest deadline 2024-09-11T08:15:39.034739+0000) 2024-09-11T08:15:56.171 INFO:journalctl@ceph.osd.7.smithi135.stdout:Sep 11 08:15:56 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-7[54744]: 2024-09-11T08:15:56.025+0000 7faedc045640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.122:6814 osd.1 since back 2024-09-11T08:15:19.035249+0000 front 2024-09-11T08:15:19.035235+0000 (oldest deadline 2024-09-11T08:15:47.835608+0000) 2024-09-11T08:15:56.171 INFO:journalctl@ceph.osd.7.smithi135.stdout:Sep 11 08:15:56 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-7[54744]: 2024-09-11T08:15:56.025+0000 7faedc045640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.122:6822 osd.2 since back 2024-09-11T08:15:29.536654+0000 front 2024-09-11T08:15:29.536686+0000 (oldest deadline 2024-09-11T08:15:53.036521+0000) 2024-09-11T08:15:56.472 INFO:journalctl@ceph.osd.5.smithi135.stdout:Sep 11 08:15:56 smithi135 podman[70288]: 2024-09-11 08:15:56.295163671 +0000 UTC m=+6.013409141 container cleanup d6e2740c975c19c040c381535a73534ee6725c60b01bfa82659f923faeeb2ae0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-5, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-11T08:15:56.472 INFO:journalctl@ceph.osd.5.smithi135.stdout:Sep 11 08:15:56 smithi135 bash[70288]: ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-5 2024-09-11T08:15:56.728 INFO:journalctl@ceph.osd.6.smithi135.stdout:Sep 11 08:15:56 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-6[48960]: 2024-09-11T08:15:56.507+0000 7fa595652640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-11T08:15:15.092751+0000 front 2024-09-11T08:15:15.093071+0000 (oldest deadline 2024-09-11T08:15:37.992793+0000) 2024-09-11T08:15:56.728 INFO:journalctl@ceph.osd.6.smithi135.stdout:Sep 11 08:15:56 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-6[48960]: 2024-09-11T08:15:56.507+0000 7fa595652640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.122:6814 osd.1 since back 2024-09-11T08:15:23.293653+0000 front 2024-09-11T08:15:23.293712+0000 (oldest deadline 2024-09-11T08:15:46.093854+0000) 2024-09-11T08:15:56.728 INFO:journalctl@ceph.osd.6.smithi135.stdout:Sep 11 08:15:56 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-6[48960]: 2024-09-11T08:15:56.507+0000 7fa595652640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.122:6822 osd.2 since back 2024-09-11T08:15:30.695537+0000 front 2024-09-11T08:15:30.695760+0000 (oldest deadline 2024-09-11T08:15:54.795367+0000) 2024-09-11T08:15:56.728 INFO:journalctl@ceph.osd.5.smithi135.stdout:Sep 11 08:15:56 smithi135 podman[70314]: 2024-09-11 08:15:56.471523461 +0000 UTC m=+0.609376209 container remove d6e2740c975c19c040c381535a73534ee6725c60b01bfa82659f923faeeb2ae0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-5, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T08:15:57.227 INFO:journalctl@ceph.osd.7.smithi135.stdout:Sep 11 08:15:57 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-7[54744]: 2024-09-11T08:15:57.059+0000 7faedc045640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-11T08:15:16.734869+0000 front 2024-09-11T08:15:16.734806+0000 (oldest deadline 2024-09-11T08:15:39.034739+0000) 2024-09-11T08:15:57.228 INFO:journalctl@ceph.osd.7.smithi135.stdout:Sep 11 08:15:57 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-7[54744]: 2024-09-11T08:15:57.059+0000 7faedc045640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.122:6814 osd.1 since back 2024-09-11T08:15:19.035249+0000 front 2024-09-11T08:15:19.035235+0000 (oldest deadline 2024-09-11T08:15:47.835608+0000) 2024-09-11T08:15:57.228 INFO:journalctl@ceph.osd.7.smithi135.stdout:Sep 11 08:15:57 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-7[54744]: 2024-09-11T08:15:57.059+0000 7faedc045640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.122:6822 osd.2 since back 2024-09-11T08:15:29.536654+0000 front 2024-09-11T08:15:29.536686+0000 (oldest deadline 2024-09-11T08:15:53.036521+0000) 2024-09-11T08:15:57.228 INFO:journalctl@ceph.osd.5.smithi135.stdout:Sep 11 08:15:56 smithi135 podman[70406]: 2024-09-11 08:15:56.816526626 +0000 UTC m=+0.016044543 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T08:15:57.228 INFO:journalctl@ceph.osd.5.smithi135.stdout:Sep 11 08:15:56 smithi135 podman[70406]: 2024-09-11 08:15:56.930645434 +0000 UTC m=+0.130163341 container create c33bd17c71e82a7649685fa20bcf51dcfd4cba49bde531b02c113c4a97bd6428 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-11T08:15:57.228 INFO:journalctl@ceph.osd.5.smithi135.stdout:Sep 11 08:15:57 smithi135 podman[70406]: 2024-09-11 08:15:57.141070957 +0000 UTC m=+0.340588872 container init c33bd17c71e82a7649685fa20bcf51dcfd4cba49bde531b02c113c4a97bd6428 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.vendor=CentOS) 2024-09-11T08:15:57.228 INFO:journalctl@ceph.osd.5.smithi135.stdout:Sep 11 08:15:57 smithi135 podman[70406]: 2024-09-11 08:15:57.144292444 +0000 UTC m=+0.343810353 container start c33bd17c71e82a7649685fa20bcf51dcfd4cba49bde531b02c113c4a97bd6428 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T08:15:57.512 INFO:journalctl@ceph.osd.5.smithi135.stdout:Sep 11 08:15:57 smithi135 podman[70406]: 2024-09-11 08:15:57.249703115 +0000 UTC m=+0.449221038 container attach c33bd17c71e82a7649685fa20bcf51dcfd4cba49bde531b02c113c4a97bd6428 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-11T08:15:57.512 INFO:journalctl@ceph.osd.5.smithi135.stdout:Sep 11 08:15:57 smithi135 podman[70406]: 2024-09-11 08:15:57.324476357 +0000 UTC m=+0.523994271 container died c33bd17c71e82a7649685fa20bcf51dcfd4cba49bde531b02c113c4a97bd6428 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-5-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-11T08:15:57.886 INFO:journalctl@ceph.osd.6.smithi135.stdout:Sep 11 08:15:57 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-6[48960]: 2024-09-11T08:15:57.511+0000 7fa595652640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-11T08:15:15.092751+0000 front 2024-09-11T08:15:15.093071+0000 (oldest deadline 2024-09-11T08:15:37.992793+0000) 2024-09-11T08:15:57.886 INFO:journalctl@ceph.osd.6.smithi135.stdout:Sep 11 08:15:57 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-6[48960]: 2024-09-11T08:15:57.511+0000 7fa595652640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.122:6814 osd.1 since back 2024-09-11T08:15:23.293653+0000 front 2024-09-11T08:15:23.293712+0000 (oldest deadline 2024-09-11T08:15:46.093854+0000) 2024-09-11T08:15:57.886 INFO:journalctl@ceph.osd.6.smithi135.stdout:Sep 11 08:15:57 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-6[48960]: 2024-09-11T08:15:57.511+0000 7fa595652640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.122:6822 osd.2 since back 2024-09-11T08:15:30.695537+0000 front 2024-09-11T08:15:30.695760+0000 (oldest deadline 2024-09-11T08:15:54.795367+0000) 2024-09-11T08:15:57.903 DEBUG:teuthology.orchestra.run.smithi135:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f1d75850-7013-11ef-bcea-c7b262605968@osd.5.service' 2024-09-11T08:15:57.944 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T08:15:57.944 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-11T08:15:57.944 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-11T08:15:57.944 DEBUG:teuthology.orchestra.run.smithi135:> sudo systemctl stop ceph-f1d75850-7013-11ef-bcea-c7b262605968@osd.6 2024-09-11T08:15:58.228 INFO:journalctl@ceph.osd.6.smithi135.stdout:Sep 11 08:15:58 smithi135 systemd[1]: Stopping Ceph osd.6 for f1d75850-7013-11ef-bcea-c7b262605968... 2024-09-11T08:15:58.228 INFO:journalctl@ceph.osd.7.smithi135.stdout:Sep 11 08:15:58 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-7[54744]: 2024-09-11T08:15:58.099+0000 7faedc045640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-11T08:15:16.734869+0000 front 2024-09-11T08:15:16.734806+0000 (oldest deadline 2024-09-11T08:15:39.034739+0000) 2024-09-11T08:15:58.228 INFO:journalctl@ceph.osd.7.smithi135.stdout:Sep 11 08:15:58 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-7[54744]: 2024-09-11T08:15:58.099+0000 7faedc045640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.122:6814 osd.1 since back 2024-09-11T08:15:19.035249+0000 front 2024-09-11T08:15:19.035235+0000 (oldest deadline 2024-09-11T08:15:47.835608+0000) 2024-09-11T08:15:58.228 INFO:journalctl@ceph.osd.7.smithi135.stdout:Sep 11 08:15:58 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-7[54744]: 2024-09-11T08:15:58.099+0000 7faedc045640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.122:6822 osd.2 since back 2024-09-11T08:15:29.536654+0000 front 2024-09-11T08:15:29.536686+0000 (oldest deadline 2024-09-11T08:15:53.036521+0000) 2024-09-11T08:15:58.728 INFO:journalctl@ceph.osd.6.smithi135.stdout:Sep 11 08:15:58 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-6[48960]: 2024-09-11T08:15:58.314+0000 7fa59a03c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T08:15:58.728 INFO:journalctl@ceph.osd.6.smithi135.stdout:Sep 11 08:15:58 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-6[48960]: 2024-09-11T08:15:58.314+0000 7fa59a03c640 -1 osd.6 66 *** Got signal Terminated *** 2024-09-11T08:15:58.728 INFO:journalctl@ceph.osd.6.smithi135.stdout:Sep 11 08:15:58 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-6[48960]: 2024-09-11T08:15:58.314+0000 7fa59a03c640 -1 osd.6 66 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T08:15:58.728 INFO:journalctl@ceph.osd.6.smithi135.stdout:Sep 11 08:15:58 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-6[48960]: 2024-09-11T08:15:58.527+0000 7fa595652640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-11T08:15:15.092751+0000 front 2024-09-11T08:15:15.093071+0000 (oldest deadline 2024-09-11T08:15:37.992793+0000) 2024-09-11T08:15:58.728 INFO:journalctl@ceph.osd.6.smithi135.stdout:Sep 11 08:15:58 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-6[48960]: 2024-09-11T08:15:58.527+0000 7fa595652640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.122:6814 osd.1 since back 2024-09-11T08:15:23.293653+0000 front 2024-09-11T08:15:23.293712+0000 (oldest deadline 2024-09-11T08:15:46.093854+0000) 2024-09-11T08:15:58.728 INFO:journalctl@ceph.osd.6.smithi135.stdout:Sep 11 08:15:58 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-6[48960]: 2024-09-11T08:15:58.527+0000 7fa595652640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.122:6822 osd.2 since back 2024-09-11T08:15:30.695537+0000 front 2024-09-11T08:15:30.695760+0000 (oldest deadline 2024-09-11T08:15:54.795367+0000) 2024-09-11T08:15:59.478 INFO:journalctl@ceph.osd.7.smithi135.stdout:Sep 11 08:15:59 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-7[54744]: 2024-09-11T08:15:59.134+0000 7faedc045640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-11T08:15:16.734869+0000 front 2024-09-11T08:15:16.734806+0000 (oldest deadline 2024-09-11T08:15:39.034739+0000) 2024-09-11T08:15:59.478 INFO:journalctl@ceph.osd.7.smithi135.stdout:Sep 11 08:15:59 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-7[54744]: 2024-09-11T08:15:59.134+0000 7faedc045640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.122:6814 osd.1 since back 2024-09-11T08:15:19.035249+0000 front 2024-09-11T08:15:19.035235+0000 (oldest deadline 2024-09-11T08:15:47.835608+0000) 2024-09-11T08:15:59.478 INFO:journalctl@ceph.osd.7.smithi135.stdout:Sep 11 08:15:59 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-7[54744]: 2024-09-11T08:15:59.134+0000 7faedc045640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.122:6822 osd.2 since back 2024-09-11T08:15:29.536654+0000 front 2024-09-11T08:15:29.536686+0000 (oldest deadline 2024-09-11T08:15:53.036521+0000) 2024-09-11T08:15:59.977 INFO:journalctl@ceph.osd.6.smithi135.stdout:Sep 11 08:15:59 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-6[48960]: 2024-09-11T08:15:59.566+0000 7fa595652640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-11T08:15:15.092751+0000 front 2024-09-11T08:15:15.093071+0000 (oldest deadline 2024-09-11T08:15:37.992793+0000) 2024-09-11T08:15:59.978 INFO:journalctl@ceph.osd.6.smithi135.stdout:Sep 11 08:15:59 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-6[48960]: 2024-09-11T08:15:59.566+0000 7fa595652640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.122:6814 osd.1 since back 2024-09-11T08:15:23.293653+0000 front 2024-09-11T08:15:23.293712+0000 (oldest deadline 2024-09-11T08:15:46.093854+0000) 2024-09-11T08:15:59.978 INFO:journalctl@ceph.osd.6.smithi135.stdout:Sep 11 08:15:59 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-6[48960]: 2024-09-11T08:15:59.566+0000 7fa595652640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.122:6822 osd.2 since back 2024-09-11T08:15:30.695537+0000 front 2024-09-11T08:15:30.695760+0000 (oldest deadline 2024-09-11T08:15:54.795367+0000) 2024-09-11T08:16:00.477 INFO:journalctl@ceph.osd.7.smithi135.stdout:Sep 11 08:16:00 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-7[54744]: 2024-09-11T08:16:00.129+0000 7faedc045640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-11T08:15:16.734869+0000 front 2024-09-11T08:15:16.734806+0000 (oldest deadline 2024-09-11T08:15:39.034739+0000) 2024-09-11T08:16:00.478 INFO:journalctl@ceph.osd.7.smithi135.stdout:Sep 11 08:16:00 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-7[54744]: 2024-09-11T08:16:00.129+0000 7faedc045640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.122:6814 osd.1 since back 2024-09-11T08:15:19.035249+0000 front 2024-09-11T08:15:19.035235+0000 (oldest deadline 2024-09-11T08:15:47.835608+0000) 2024-09-11T08:16:00.478 INFO:journalctl@ceph.osd.7.smithi135.stdout:Sep 11 08:16:00 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-7[54744]: 2024-09-11T08:16:00.129+0000 7faedc045640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.122:6822 osd.2 since back 2024-09-11T08:15:29.536654+0000 front 2024-09-11T08:15:29.536686+0000 (oldest deadline 2024-09-11T08:15:53.036521+0000) 2024-09-11T08:16:00.978 INFO:journalctl@ceph.osd.6.smithi135.stdout:Sep 11 08:16:00 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-6[48960]: 2024-09-11T08:16:00.536+0000 7fa595652640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-11T08:15:15.092751+0000 front 2024-09-11T08:15:15.093071+0000 (oldest deadline 2024-09-11T08:15:37.992793+0000) 2024-09-11T08:16:00.978 INFO:journalctl@ceph.osd.6.smithi135.stdout:Sep 11 08:16:00 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-6[48960]: 2024-09-11T08:16:00.536+0000 7fa595652640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.122:6814 osd.1 since back 2024-09-11T08:15:23.293653+0000 front 2024-09-11T08:15:23.293712+0000 (oldest deadline 2024-09-11T08:15:46.093854+0000) 2024-09-11T08:16:00.978 INFO:journalctl@ceph.osd.6.smithi135.stdout:Sep 11 08:16:00 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-6[48960]: 2024-09-11T08:16:00.536+0000 7fa595652640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.122:6822 osd.2 since back 2024-09-11T08:15:30.695537+0000 front 2024-09-11T08:15:30.695760+0000 (oldest deadline 2024-09-11T08:15:54.795367+0000) 2024-09-11T08:16:01.477 INFO:journalctl@ceph.osd.7.smithi135.stdout:Sep 11 08:16:01 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-7[54744]: 2024-09-11T08:16:01.135+0000 7faedc045640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-11T08:15:16.734869+0000 front 2024-09-11T08:15:16.734806+0000 (oldest deadline 2024-09-11T08:15:39.034739+0000) 2024-09-11T08:16:01.478 INFO:journalctl@ceph.osd.7.smithi135.stdout:Sep 11 08:16:01 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-7[54744]: 2024-09-11T08:16:01.135+0000 7faedc045640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.122:6814 osd.1 since back 2024-09-11T08:15:19.035249+0000 front 2024-09-11T08:15:19.035235+0000 (oldest deadline 2024-09-11T08:15:47.835608+0000) 2024-09-11T08:16:01.478 INFO:journalctl@ceph.osd.7.smithi135.stdout:Sep 11 08:16:01 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-7[54744]: 2024-09-11T08:16:01.135+0000 7faedc045640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.122:6822 osd.2 since back 2024-09-11T08:15:29.536654+0000 front 2024-09-11T08:15:29.536686+0000 (oldest deadline 2024-09-11T08:15:53.036521+0000) 2024-09-11T08:16:01.978 INFO:journalctl@ceph.osd.6.smithi135.stdout:Sep 11 08:16:01 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-6[48960]: 2024-09-11T08:16:01.488+0000 7fa595652640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-11T08:15:15.092751+0000 front 2024-09-11T08:15:15.093071+0000 (oldest deadline 2024-09-11T08:15:37.992793+0000) 2024-09-11T08:16:01.978 INFO:journalctl@ceph.osd.6.smithi135.stdout:Sep 11 08:16:01 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-6[48960]: 2024-09-11T08:16:01.488+0000 7fa595652640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.122:6814 osd.1 since back 2024-09-11T08:15:23.293653+0000 front 2024-09-11T08:15:23.293712+0000 (oldest deadline 2024-09-11T08:15:46.093854+0000) 2024-09-11T08:16:01.978 INFO:journalctl@ceph.osd.6.smithi135.stdout:Sep 11 08:16:01 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-6[48960]: 2024-09-11T08:16:01.488+0000 7fa595652640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.122:6822 osd.2 since back 2024-09-11T08:15:30.695537+0000 front 2024-09-11T08:15:30.695760+0000 (oldest deadline 2024-09-11T08:15:54.795367+0000) 2024-09-11T08:16:02.477 INFO:journalctl@ceph.osd.7.smithi135.stdout:Sep 11 08:16:02 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-7[54744]: 2024-09-11T08:16:02.172+0000 7faedc045640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-11T08:15:16.734869+0000 front 2024-09-11T08:15:16.734806+0000 (oldest deadline 2024-09-11T08:15:39.034739+0000) 2024-09-11T08:16:02.478 INFO:journalctl@ceph.osd.7.smithi135.stdout:Sep 11 08:16:02 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-7[54744]: 2024-09-11T08:16:02.172+0000 7faedc045640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.122:6814 osd.1 since back 2024-09-11T08:15:19.035249+0000 front 2024-09-11T08:15:19.035235+0000 (oldest deadline 2024-09-11T08:15:47.835608+0000) 2024-09-11T08:16:02.478 INFO:journalctl@ceph.osd.7.smithi135.stdout:Sep 11 08:16:02 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-7[54744]: 2024-09-11T08:16:02.172+0000 7faedc045640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.122:6822 osd.2 since back 2024-09-11T08:15:29.536654+0000 front 2024-09-11T08:15:29.536686+0000 (oldest deadline 2024-09-11T08:15:53.036521+0000) 2024-09-11T08:16:02.978 INFO:journalctl@ceph.osd.6.smithi135.stdout:Sep 11 08:16:02 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-6[48960]: 2024-09-11T08:16:02.486+0000 7fa595652640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-11T08:15:15.092751+0000 front 2024-09-11T08:15:15.093071+0000 (oldest deadline 2024-09-11T08:15:37.992793+0000) 2024-09-11T08:16:02.978 INFO:journalctl@ceph.osd.6.smithi135.stdout:Sep 11 08:16:02 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-6[48960]: 2024-09-11T08:16:02.486+0000 7fa595652640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.122:6814 osd.1 since back 2024-09-11T08:15:23.293653+0000 front 2024-09-11T08:15:23.293712+0000 (oldest deadline 2024-09-11T08:15:46.093854+0000) 2024-09-11T08:16:02.978 INFO:journalctl@ceph.osd.6.smithi135.stdout:Sep 11 08:16:02 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-6[48960]: 2024-09-11T08:16:02.486+0000 7fa595652640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.122:6822 osd.2 since back 2024-09-11T08:15:30.695537+0000 front 2024-09-11T08:15:30.695760+0000 (oldest deadline 2024-09-11T08:15:54.795367+0000) 2024-09-11T08:16:02.978 INFO:journalctl@ceph.osd.6.smithi135.stdout:Sep 11 08:16:02 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-6[48960]: 2024-09-11T08:16:02.486+0000 7fa595652640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.122:6830 osd.3 since back 2024-09-11T08:15:39.996572+0000 front 2024-09-11T08:15:39.996559+0000 (oldest deadline 2024-09-11T08:16:01.696398+0000) 2024-09-11T08:16:03.478 INFO:journalctl@ceph.osd.7.smithi135.stdout:Sep 11 08:16:03 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-7[54744]: 2024-09-11T08:16:03.166+0000 7faedc045640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-11T08:15:16.734869+0000 front 2024-09-11T08:15:16.734806+0000 (oldest deadline 2024-09-11T08:15:39.034739+0000) 2024-09-11T08:16:03.478 INFO:journalctl@ceph.osd.7.smithi135.stdout:Sep 11 08:16:03 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-7[54744]: 2024-09-11T08:16:03.166+0000 7faedc045640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.122:6814 osd.1 since back 2024-09-11T08:15:19.035249+0000 front 2024-09-11T08:15:19.035235+0000 (oldest deadline 2024-09-11T08:15:47.835608+0000) 2024-09-11T08:16:03.478 INFO:journalctl@ceph.osd.7.smithi135.stdout:Sep 11 08:16:03 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-7[54744]: 2024-09-11T08:16:03.166+0000 7faedc045640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.122:6822 osd.2 since back 2024-09-11T08:15:29.536654+0000 front 2024-09-11T08:15:29.536686+0000 (oldest deadline 2024-09-11T08:15:53.036521+0000) 2024-09-11T08:16:03.478 INFO:journalctl@ceph.osd.7.smithi135.stdout:Sep 11 08:16:03 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-7[54744]: 2024-09-11T08:16:03.166+0000 7faedc045640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.122:6830 osd.3 since back 2024-09-11T08:15:37.737538+0000 front 2024-09-11T08:15:37.737497+0000 (oldest deadline 2024-09-11T08:16:02.437432+0000) 2024-09-11T08:16:04.048 INFO:journalctl@ceph.osd.6.smithi135.stdout:Sep 11 08:16:03 smithi135 podman[70508]: 2024-09-11 08:16:03.779106851 +0000 UTC m=+5.612393460 container died f455f6660ff1cf992cf4a4a5790022695aa1ac281f76ce7eafc5580ab41822b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-6, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-11T08:16:04.449 INFO:journalctl@ceph.osd.6.smithi135.stdout:Sep 11 08:16:04 smithi135 podman[70508]: 2024-09-11 08:16:04.147553185 +0000 UTC m=+5.980839792 container cleanup f455f6660ff1cf992cf4a4a5790022695aa1ac281f76ce7eafc5580ab41822b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-6, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828) 2024-09-11T08:16:04.449 INFO:journalctl@ceph.osd.6.smithi135.stdout:Sep 11 08:16:04 smithi135 bash[70508]: ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-6 2024-09-11T08:16:04.449 INFO:journalctl@ceph.osd.7.smithi135.stdout:Sep 11 08:16:04 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-7[54744]: 2024-09-11T08:16:04.156+0000 7faedc045640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-11T08:15:16.734869+0000 front 2024-09-11T08:15:16.734806+0000 (oldest deadline 2024-09-11T08:15:39.034739+0000) 2024-09-11T08:16:04.449 INFO:journalctl@ceph.osd.7.smithi135.stdout:Sep 11 08:16:04 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-7[54744]: 2024-09-11T08:16:04.157+0000 7faedc045640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.122:6814 osd.1 since back 2024-09-11T08:15:19.035249+0000 front 2024-09-11T08:15:19.035235+0000 (oldest deadline 2024-09-11T08:15:47.835608+0000) 2024-09-11T08:16:04.449 INFO:journalctl@ceph.osd.7.smithi135.stdout:Sep 11 08:16:04 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-7[54744]: 2024-09-11T08:16:04.157+0000 7faedc045640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.122:6822 osd.2 since back 2024-09-11T08:15:29.536654+0000 front 2024-09-11T08:15:29.536686+0000 (oldest deadline 2024-09-11T08:15:53.036521+0000) 2024-09-11T08:16:04.450 INFO:journalctl@ceph.osd.7.smithi135.stdout:Sep 11 08:16:04 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-7[54744]: 2024-09-11T08:16:04.157+0000 7faedc045640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.122:6830 osd.3 since back 2024-09-11T08:15:37.737538+0000 front 2024-09-11T08:15:37.737497+0000 (oldest deadline 2024-09-11T08:16:02.437432+0000) 2024-09-11T08:16:04.727 INFO:journalctl@ceph.osd.6.smithi135.stdout:Sep 11 08:16:04 smithi135 podman[70524]: 2024-09-11 08:16:04.449091975 +0000 UTC m=+0.661672372 container remove f455f6660ff1cf992cf4a4a5790022695aa1ac281f76ce7eafc5580ab41822b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-6, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T08:16:05.077 INFO:journalctl@ceph.osd.6.smithi135.stdout:Sep 11 08:16:04 smithi135 podman[70616]: 2024-09-11 08:16:04.671532709 +0000 UTC m=+0.015742725 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T08:16:05.077 INFO:journalctl@ceph.osd.6.smithi135.stdout:Sep 11 08:16:04 smithi135 podman[70616]: 2024-09-11 08:16:04.774915424 +0000 UTC m=+0.119125434 container create e200e6ab1e9c4ce2c1b20b129503ce28289f4d0ffc5fa1718647dd80d8b8a4aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-6-deactivate, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-11T08:16:05.077 INFO:journalctl@ceph.osd.6.smithi135.stdout:Sep 11 08:16:04 smithi135 podman[70616]: 2024-09-11 08:16:04.968835697 +0000 UTC m=+0.313045708 container init e200e6ab1e9c4ce2c1b20b129503ce28289f4d0ffc5fa1718647dd80d8b8a4aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux ) 2024-09-11T08:16:05.077 INFO:journalctl@ceph.osd.6.smithi135.stdout:Sep 11 08:16:04 smithi135 podman[70616]: 2024-09-11 08:16:04.971972225 +0000 UTC m=+0.316182237 container start e200e6ab1e9c4ce2c1b20b129503ce28289f4d0ffc5fa1718647dd80d8b8a4aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-11T08:16:05.077 INFO:journalctl@ceph.osd.6.smithi135.stdout:Sep 11 08:16:05 smithi135 podman[70616]: 2024-09-11 08:16:05.077014283 +0000 UTC m=+0.421224312 container attach e200e6ab1e9c4ce2c1b20b129503ce28289f4d0ffc5fa1718647dd80d8b8a4aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-6-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-09-11T08:16:05.447 INFO:journalctl@ceph.osd.6.smithi135.stdout:Sep 11 08:16:05 smithi135 podman[70616]: 2024-09-11 08:16:05.156576267 +0000 UTC m=+0.500786279 container died e200e6ab1e9c4ce2c1b20b129503ce28289f4d0ffc5fa1718647dd80d8b8a4aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS) 2024-09-11T08:16:05.515 INFO:journalctl@ceph.osd.7.smithi135.stdout:Sep 11 08:16:05 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-7[54744]: 2024-09-11T08:16:05.206+0000 7faedc045640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-11T08:15:16.734869+0000 front 2024-09-11T08:15:16.734806+0000 (oldest deadline 2024-09-11T08:15:39.034739+0000) 2024-09-11T08:16:05.515 INFO:journalctl@ceph.osd.7.smithi135.stdout:Sep 11 08:16:05 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-7[54744]: 2024-09-11T08:16:05.206+0000 7faedc045640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.122:6814 osd.1 since back 2024-09-11T08:15:19.035249+0000 front 2024-09-11T08:15:19.035235+0000 (oldest deadline 2024-09-11T08:15:47.835608+0000) 2024-09-11T08:16:05.516 INFO:journalctl@ceph.osd.7.smithi135.stdout:Sep 11 08:16:05 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-7[54744]: 2024-09-11T08:16:05.206+0000 7faedc045640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.122:6822 osd.2 since back 2024-09-11T08:15:29.536654+0000 front 2024-09-11T08:15:29.536686+0000 (oldest deadline 2024-09-11T08:15:53.036521+0000) 2024-09-11T08:16:05.516 INFO:journalctl@ceph.osd.7.smithi135.stdout:Sep 11 08:16:05 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-7[54744]: 2024-09-11T08:16:05.206+0000 7faedc045640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.122:6830 osd.3 since back 2024-09-11T08:15:37.737538+0000 front 2024-09-11T08:15:37.737497+0000 (oldest deadline 2024-09-11T08:16:02.437432+0000) 2024-09-11T08:16:05.782 DEBUG:teuthology.orchestra.run.smithi135:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f1d75850-7013-11ef-bcea-c7b262605968@osd.6.service' 2024-09-11T08:16:05.821 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T08:16:05.821 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-11T08:16:05.821 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-11T08:16:05.821 DEBUG:teuthology.orchestra.run.smithi135:> sudo systemctl stop ceph-f1d75850-7013-11ef-bcea-c7b262605968@osd.7 2024-09-11T08:16:06.150 INFO:journalctl@ceph.osd.7.smithi135.stdout:Sep 11 08:16:05 smithi135 systemd[1]: Stopping Ceph osd.7 for f1d75850-7013-11ef-bcea-c7b262605968... 2024-09-11T08:16:06.151 INFO:journalctl@ceph.osd.7.smithi135.stdout:Sep 11 08:16:06 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-7[54744]: 2024-09-11T08:16:06.149+0000 7faee0a2f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T08:16:06.151 INFO:journalctl@ceph.osd.7.smithi135.stdout:Sep 11 08:16:06 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-7[54744]: 2024-09-11T08:16:06.149+0000 7faee0a2f640 -1 osd.7 66 *** Got signal Terminated *** 2024-09-11T08:16:06.477 INFO:journalctl@ceph.osd.7.smithi135.stdout:Sep 11 08:16:06 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-7[54744]: 2024-09-11T08:16:06.149+0000 7faee0a2f640 -1 osd.7 66 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T08:16:06.478 INFO:journalctl@ceph.osd.7.smithi135.stdout:Sep 11 08:16:06 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-7[54744]: 2024-09-11T08:16:06.250+0000 7faedc045640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-11T08:15:16.734869+0000 front 2024-09-11T08:15:16.734806+0000 (oldest deadline 2024-09-11T08:15:39.034739+0000) 2024-09-11T08:16:06.478 INFO:journalctl@ceph.osd.7.smithi135.stdout:Sep 11 08:16:06 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-7[54744]: 2024-09-11T08:16:06.250+0000 7faedc045640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.122:6814 osd.1 since back 2024-09-11T08:15:19.035249+0000 front 2024-09-11T08:15:19.035235+0000 (oldest deadline 2024-09-11T08:15:47.835608+0000) 2024-09-11T08:16:06.478 INFO:journalctl@ceph.osd.7.smithi135.stdout:Sep 11 08:16:06 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-7[54744]: 2024-09-11T08:16:06.250+0000 7faedc045640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.122:6822 osd.2 since back 2024-09-11T08:15:29.536654+0000 front 2024-09-11T08:15:29.536686+0000 (oldest deadline 2024-09-11T08:15:53.036521+0000) 2024-09-11T08:16:06.478 INFO:journalctl@ceph.osd.7.smithi135.stdout:Sep 11 08:16:06 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-7[54744]: 2024-09-11T08:16:06.250+0000 7faedc045640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.122:6830 osd.3 since back 2024-09-11T08:15:37.737538+0000 front 2024-09-11T08:15:37.737497+0000 (oldest deadline 2024-09-11T08:16:02.437432+0000) 2024-09-11T08:16:07.728 INFO:journalctl@ceph.osd.7.smithi135.stdout:Sep 11 08:16:07 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-7[54744]: 2024-09-11T08:16:07.293+0000 7faedc045640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-11T08:15:16.734869+0000 front 2024-09-11T08:15:16.734806+0000 (oldest deadline 2024-09-11T08:15:39.034739+0000) 2024-09-11T08:16:07.728 INFO:journalctl@ceph.osd.7.smithi135.stdout:Sep 11 08:16:07 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-7[54744]: 2024-09-11T08:16:07.293+0000 7faedc045640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.122:6814 osd.1 since back 2024-09-11T08:15:19.035249+0000 front 2024-09-11T08:15:19.035235+0000 (oldest deadline 2024-09-11T08:15:47.835608+0000) 2024-09-11T08:16:07.728 INFO:journalctl@ceph.osd.7.smithi135.stdout:Sep 11 08:16:07 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-7[54744]: 2024-09-11T08:16:07.293+0000 7faedc045640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.122:6822 osd.2 since back 2024-09-11T08:15:29.536654+0000 front 2024-09-11T08:15:29.536686+0000 (oldest deadline 2024-09-11T08:15:53.036521+0000) 2024-09-11T08:16:07.728 INFO:journalctl@ceph.osd.7.smithi135.stdout:Sep 11 08:16:07 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-7[54744]: 2024-09-11T08:16:07.293+0000 7faedc045640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.122:6830 osd.3 since back 2024-09-11T08:15:37.737538+0000 front 2024-09-11T08:15:37.737497+0000 (oldest deadline 2024-09-11T08:16:02.437432+0000) 2024-09-11T08:16:08.629 INFO:journalctl@ceph.osd.7.smithi135.stdout:Sep 11 08:16:08 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-7[54744]: 2024-09-11T08:16:08.305+0000 7faedc045640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-11T08:15:16.734869+0000 front 2024-09-11T08:15:16.734806+0000 (oldest deadline 2024-09-11T08:15:39.034739+0000) 2024-09-11T08:16:08.629 INFO:journalctl@ceph.osd.7.smithi135.stdout:Sep 11 08:16:08 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-7[54744]: 2024-09-11T08:16:08.305+0000 7faedc045640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.122:6814 osd.1 since back 2024-09-11T08:15:19.035249+0000 front 2024-09-11T08:15:19.035235+0000 (oldest deadline 2024-09-11T08:15:47.835608+0000) 2024-09-11T08:16:08.629 INFO:journalctl@ceph.osd.7.smithi135.stdout:Sep 11 08:16:08 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-7[54744]: 2024-09-11T08:16:08.305+0000 7faedc045640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.122:6822 osd.2 since back 2024-09-11T08:15:29.536654+0000 front 2024-09-11T08:15:29.536686+0000 (oldest deadline 2024-09-11T08:15:53.036521+0000) 2024-09-11T08:16:08.629 INFO:journalctl@ceph.osd.7.smithi135.stdout:Sep 11 08:16:08 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-7[54744]: 2024-09-11T08:16:08.305+0000 7faedc045640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.122:6830 osd.3 since back 2024-09-11T08:15:37.737538+0000 front 2024-09-11T08:15:37.737497+0000 (oldest deadline 2024-09-11T08:16:02.437432+0000) 2024-09-11T08:16:09.728 INFO:journalctl@ceph.osd.7.smithi135.stdout:Sep 11 08:16:09 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-7[54744]: 2024-09-11T08:16:09.281+0000 7faedc045640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-11T08:15:16.734869+0000 front 2024-09-11T08:15:16.734806+0000 (oldest deadline 2024-09-11T08:15:39.034739+0000) 2024-09-11T08:16:09.728 INFO:journalctl@ceph.osd.7.smithi135.stdout:Sep 11 08:16:09 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-7[54744]: 2024-09-11T08:16:09.281+0000 7faedc045640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.122:6814 osd.1 since back 2024-09-11T08:15:19.035249+0000 front 2024-09-11T08:15:19.035235+0000 (oldest deadline 2024-09-11T08:15:47.835608+0000) 2024-09-11T08:16:09.728 INFO:journalctl@ceph.osd.7.smithi135.stdout:Sep 11 08:16:09 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-7[54744]: 2024-09-11T08:16:09.281+0000 7faedc045640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.122:6822 osd.2 since back 2024-09-11T08:15:29.536654+0000 front 2024-09-11T08:15:29.536686+0000 (oldest deadline 2024-09-11T08:15:53.036521+0000) 2024-09-11T08:16:09.728 INFO:journalctl@ceph.osd.7.smithi135.stdout:Sep 11 08:16:09 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-7[54744]: 2024-09-11T08:16:09.281+0000 7faedc045640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.122:6830 osd.3 since back 2024-09-11T08:15:37.737538+0000 front 2024-09-11T08:15:37.737497+0000 (oldest deadline 2024-09-11T08:16:02.437432+0000) 2024-09-11T08:16:09.728 INFO:journalctl@ceph.osd.7.smithi135.stdout:Sep 11 08:16:09 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-7[54744]: 2024-09-11T08:16:09.281+0000 7faedc045640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.135:6804 osd.4 since back 2024-09-11T08:15:42.437863+0000 front 2024-09-11T08:15:42.437936+0000 (oldest deadline 2024-09-11T08:16:08.337869+0000) 2024-09-11T08:16:10.728 INFO:journalctl@ceph.osd.7.smithi135.stdout:Sep 11 08:16:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-7[54744]: 2024-09-11T08:16:10.318+0000 7faedc045640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-11T08:15:16.734869+0000 front 2024-09-11T08:15:16.734806+0000 (oldest deadline 2024-09-11T08:15:39.034739+0000) 2024-09-11T08:16:10.728 INFO:journalctl@ceph.osd.7.smithi135.stdout:Sep 11 08:16:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-7[54744]: 2024-09-11T08:16:10.318+0000 7faedc045640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.122:6814 osd.1 since back 2024-09-11T08:15:19.035249+0000 front 2024-09-11T08:15:19.035235+0000 (oldest deadline 2024-09-11T08:15:47.835608+0000) 2024-09-11T08:16:10.729 INFO:journalctl@ceph.osd.7.smithi135.stdout:Sep 11 08:16:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-7[54744]: 2024-09-11T08:16:10.318+0000 7faedc045640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.122:6822 osd.2 since back 2024-09-11T08:15:29.536654+0000 front 2024-09-11T08:15:29.536686+0000 (oldest deadline 2024-09-11T08:15:53.036521+0000) 2024-09-11T08:16:10.729 INFO:journalctl@ceph.osd.7.smithi135.stdout:Sep 11 08:16:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-7[54744]: 2024-09-11T08:16:10.318+0000 7faedc045640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.122:6830 osd.3 since back 2024-09-11T08:15:37.737538+0000 front 2024-09-11T08:15:37.737497+0000 (oldest deadline 2024-09-11T08:16:02.437432+0000) 2024-09-11T08:16:10.729 INFO:journalctl@ceph.osd.7.smithi135.stdout:Sep 11 08:16:10 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-7[54744]: 2024-09-11T08:16:10.318+0000 7faedc045640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.135:6804 osd.4 since back 2024-09-11T08:15:42.437863+0000 front 2024-09-11T08:15:42.437936+0000 (oldest deadline 2024-09-11T08:16:08.337869+0000) 2024-09-11T08:16:11.791 INFO:journalctl@ceph.osd.7.smithi135.stdout:Sep 11 08:16:11 smithi135 podman[70716]: 2024-09-11 08:16:11.478247611 +0000 UTC m=+5.432345554 container died 4a8a5a20d4f131dcfe653db1061eae136384b8390c618b2f1fc4893f51206c17 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-7, ceph=True, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux ) 2024-09-11T08:16:12.228 INFO:journalctl@ceph.osd.7.smithi135.stdout:Sep 11 08:16:11 smithi135 podman[70716]: 2024-09-11 08:16:11.79095633 +0000 UTC m=+5.745054271 container cleanup 4a8a5a20d4f131dcfe653db1061eae136384b8390c618b2f1fc4893f51206c17 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-7, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, ceph=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-11T08:16:12.228 INFO:journalctl@ceph.osd.7.smithi135.stdout:Sep 11 08:16:11 smithi135 bash[70716]: ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-7 2024-09-11T08:16:12.228 INFO:journalctl@ceph.osd.7.smithi135.stdout:Sep 11 08:16:11 smithi135 podman[70743]: 2024-09-11 08:16:11.933759168 +0000 UTC m=+0.450909738 container remove 4a8a5a20d4f131dcfe653db1061eae136384b8390c618b2f1fc4893f51206c17 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-7, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-11T08:16:12.628 INFO:journalctl@ceph.osd.7.smithi135.stdout:Sep 11 08:16:12 smithi135 podman[70833]: 2024-09-11 08:16:12.334433107 +0000 UTC m=+0.096402288 container create aa64c33ff21cdf472d102c46fe7526c2759d87361c08c4c38d7c6d58ef7b3abd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T08:16:12.628 INFO:journalctl@ceph.osd.7.smithi135.stdout:Sep 11 08:16:12 smithi135 podman[70833]: 2024-09-11 08:16:12.254208165 +0000 UTC m=+0.016177360 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T08:16:12.628 INFO:journalctl@ceph.osd.7.smithi135.stdout:Sep 11 08:16:12 smithi135 podman[70833]: 2024-09-11 08:16:12.545022902 +0000 UTC m=+0.306992099 container init aa64c33ff21cdf472d102c46fe7526c2759d87361c08c4c38d7c6d58ef7b3abd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-11T08:16:12.628 INFO:journalctl@ceph.osd.7.smithi135.stdout:Sep 11 08:16:12 smithi135 podman[70833]: 2024-09-11 08:16:12.548388763 +0000 UTC m=+0.310357952 container start aa64c33ff21cdf472d102c46fe7526c2759d87361c08c4c38d7c6d58ef7b3abd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True) 2024-09-11T08:16:12.629 INFO:journalctl@ceph.osd.7.smithi135.stdout:Sep 11 08:16:12 smithi135 podman[70833]: 2024-09-11 08:16:12.628235337 +0000 UTC m=+0.390204522 container attach aa64c33ff21cdf472d102c46fe7526c2759d87361c08c4c38d7c6d58ef7b3abd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-7-deactivate, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-11T08:16:12.978 INFO:journalctl@ceph.osd.7.smithi135.stdout:Sep 11 08:16:12 smithi135 podman[70833]: 2024-09-11 08:16:12.725774313 +0000 UTC m=+0.487743511 container died aa64c33ff21cdf472d102c46fe7526c2759d87361c08c4c38d7c6d58ef7b3abd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_CLEAN=True) 2024-09-11T08:16:13.320 DEBUG:teuthology.orchestra.run.smithi135:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f1d75850-7013-11ef-bcea-c7b262605968@osd.7.service' 2024-09-11T08:16:13.359 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T08:16:13.359 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-11T08:16:13.359 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopping rgw.foo.a... 2024-09-11T08:16:13.359 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ceph-f1d75850-7013-11ef-bcea-c7b262605968@rgw.foo.a 2024-09-11T08:16:13.733 INFO:journalctl@ceph.rgw.foo.a.smithi122.stdout:Sep 11 08:16:13 smithi122 systemd[1]: Stopping Ceph rgw.foo.a for f1d75850-7013-11ef-bcea-c7b262605968... 2024-09-11T08:16:14.070 INFO:journalctl@ceph.rgw.foo.a.smithi122.stdout:Sep 11 08:16:13 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-rgw-foo-a[65607]: 2024-09-11T08:16:13.742+0000 7fe10d713640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/radosgw -n client.rgw.foo.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T08:16:14.070 INFO:journalctl@ceph.rgw.foo.a.smithi122.stdout:Sep 11 08:16:13 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-rgw-foo-a[65607]: 2024-09-11T08:16:13.742+0000 7fe111c1b7c0 -1 shutting down 2024-09-11T08:16:24.110 INFO:journalctl@ceph.rgw.foo.a.smithi122.stdout:Sep 11 08:16:23 smithi122 bash[83008]: time="2024-09-11T08:16:23Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-f1d75850-7013-11ef-bcea-c7b262605968-rgw-foo-a in 10 seconds, resorting to SIGKILL" 2024-09-11T08:16:24.111 INFO:journalctl@ceph.rgw.foo.a.smithi122.stdout:Sep 11 08:16:23 smithi122 podman[83008]: 2024-09-11 08:16:23.757607961 +0000 UTC m=+10.210574772 container died a6a48059e8e9b2c7474e4034debb10ac18ad7d73a6e857ad49da141ea9f1468a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-rgw-foo-a, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-11T08:16:24.394 INFO:journalctl@ceph.rgw.foo.a.smithi122.stdout:Sep 11 08:16:24 smithi122 podman[83008]: 2024-09-11 08:16:24.200885339 +0000 UTC m=+10.653852145 container cleanup a6a48059e8e9b2c7474e4034debb10ac18ad7d73a6e857ad49da141ea9f1468a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-rgw-foo-a, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, io.buildah.version=1.37.2) 2024-09-11T08:16:24.394 INFO:journalctl@ceph.rgw.foo.a.smithi122.stdout:Sep 11 08:16:24 smithi122 bash[83008]: ceph-f1d75850-7013-11ef-bcea-c7b262605968-rgw-foo-a 2024-09-11T08:16:24.629 DEBUG:teuthology.orchestra.run.smithi122:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f1d75850-7013-11ef-bcea-c7b262605968@rgw.foo.a.service' 2024-09-11T08:16:24.651 INFO:journalctl@ceph.rgw.foo.a.smithi122.stdout:Sep 11 08:16:24 smithi122 podman[83032]: 2024-09-11 08:16:24.394056907 +0000 UTC m=+0.634463615 container remove a6a48059e8e9b2c7474e4034debb10ac18ad7d73a6e857ad49da141ea9f1468a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-rgw-foo-a, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, maintainer=Guillaume Abrioux ) 2024-09-11T08:16:24.651 INFO:journalctl@ceph.rgw.foo.a.smithi122.stdout:Sep 11 08:16:24 smithi122 systemd[1]: ceph-f1d75850-7013-11ef-bcea-c7b262605968@rgw.foo.a.service: Main process exited, code=exited, status=137/n/a 2024-09-11T08:16:24.652 INFO:journalctl@ceph.rgw.foo.a.smithi122.stdout:Sep 11 08:16:24 smithi122 systemd[1]: ceph-f1d75850-7013-11ef-bcea-c7b262605968@rgw.foo.a.service: Failed with result 'exit-code'. 2024-09-11T08:16:24.652 INFO:journalctl@ceph.rgw.foo.a.smithi122.stdout:Sep 11 08:16:24 smithi122 systemd[1]: Stopped Ceph rgw.foo.a for f1d75850-7013-11ef-bcea-c7b262605968. 2024-09-11T08:16:24.652 INFO:journalctl@ceph.rgw.foo.a.smithi122.stdout:Sep 11 08:16:24 smithi122 systemd[1]: ceph-f1d75850-7013-11ef-bcea-c7b262605968@rgw.foo.a.service: Consumed 2.145s CPU time. 2024-09-11T08:16:24.665 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T08:16:24.665 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopped rgw.foo.a 2024-09-11T08:16:24.665 INFO:tasks.cephadm.prometheus.a:Stopping prometheus.a... 2024-09-11T08:16:24.665 DEBUG:teuthology.orchestra.run.smithi135:> sudo systemctl stop ceph-f1d75850-7013-11ef-bcea-c7b262605968@prometheus.a 2024-09-11T08:16:24.967 INFO:journalctl@ceph.prometheus.a.smithi135.stdout:Sep 11 08:16:24 smithi135 systemd[1]: Stopping Ceph prometheus.a for f1d75850-7013-11ef-bcea-c7b262605968... 2024-09-11T08:16:25.228 INFO:journalctl@ceph.prometheus.a.smithi135.stdout:Sep 11 08:16:24 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-prometheus-a[64624]: ts=2024-09-11T08:16:24.966Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-09-11T08:16:25.228 INFO:journalctl@ceph.prometheus.a.smithi135.stdout:Sep 11 08:16:24 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-prometheus-a[64624]: ts=2024-09-11T08:16:24.967Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-09-11T08:16:25.228 INFO:journalctl@ceph.prometheus.a.smithi135.stdout:Sep 11 08:16:24 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-prometheus-a[64624]: ts=2024-09-11T08:16:24.967Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-09-11T08:16:25.228 INFO:journalctl@ceph.prometheus.a.smithi135.stdout:Sep 11 08:16:24 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-prometheus-a[64624]: ts=2024-09-11T08:16:24.967Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-09-11T08:16:25.228 INFO:journalctl@ceph.prometheus.a.smithi135.stdout:Sep 11 08:16:24 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-prometheus-a[64624]: ts=2024-09-11T08:16:24.967Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-09-11T08:16:25.229 INFO:journalctl@ceph.prometheus.a.smithi135.stdout:Sep 11 08:16:24 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-prometheus-a[64624]: ts=2024-09-11T08:16:24.967Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-09-11T08:16:25.229 INFO:journalctl@ceph.prometheus.a.smithi135.stdout:Sep 11 08:16:24 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-prometheus-a[64624]: ts=2024-09-11T08:16:24.967Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-09-11T08:16:25.229 INFO:journalctl@ceph.prometheus.a.smithi135.stdout:Sep 11 08:16:24 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-prometheus-a[64624]: ts=2024-09-11T08:16:24.967Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-09-11T08:16:25.229 INFO:journalctl@ceph.prometheus.a.smithi135.stdout:Sep 11 08:16:24 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-prometheus-a[64624]: ts=2024-09-11T08:16:24.967Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-09-11T08:16:25.229 INFO:journalctl@ceph.prometheus.a.smithi135.stdout:Sep 11 08:16:24 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-prometheus-a[64624]: ts=2024-09-11T08:16:24.967Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-09-11T08:16:25.229 INFO:journalctl@ceph.prometheus.a.smithi135.stdout:Sep 11 08:16:24 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-prometheus-a[64624]: ts=2024-09-11T08:16:24.967Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-09-11T08:16:25.229 INFO:journalctl@ceph.prometheus.a.smithi135.stdout:Sep 11 08:16:24 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-prometheus-a[64624]: ts=2024-09-11T08:16:24.967Z caller=main.go:1147 level=info msg="See you next time!" 2024-09-11T08:16:25.229 INFO:journalctl@ceph.prometheus.a.smithi135.stdout:Sep 11 08:16:24 smithi135 podman[70936]: 2024-09-11 08:16:24.977474033 +0000 UTC m=+0.116767640 container died faf0895c4ca57d977ae69ff6f4c21fa41b6054eda0ba470e7d049d47bc802118 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T08:16:25.592 INFO:journalctl@ceph.prometheus.a.smithi135.stdout:Sep 11 08:16:25 smithi135 podman[70936]: 2024-09-11 08:16:25.324828588 +0000 UTC m=+0.464122182 container cleanup faf0895c4ca57d977ae69ff6f4c21fa41b6054eda0ba470e7d049d47bc802118 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T08:16:25.592 INFO:journalctl@ceph.prometheus.a.smithi135.stdout:Sep 11 08:16:25 smithi135 bash[70936]: ceph-f1d75850-7013-11ef-bcea-c7b262605968-prometheus-a 2024-09-11T08:16:25.752 DEBUG:teuthology.orchestra.run.smithi135:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f1d75850-7013-11ef-bcea-c7b262605968@prometheus.a.service' 2024-09-11T08:16:25.788 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T08:16:25.788 INFO:tasks.cephadm.prometheus.a:Stopped prometheus.a 2024-09-11T08:16:25.788 DEBUG:teuthology.orchestra.run.smithi122:> sudo cephadm rm-cluster --fsid f1d75850-7013-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-11T08:16:27.923 INFO:journalctl@ceph.alertmanager.a.smithi122.stdout:Sep 11 08:16:27 smithi122 systemd[1]: Stopping Ceph alertmanager.a for f1d75850-7013-11ef-bcea-c7b262605968... 2024-09-11T08:16:28.207 INFO:journalctl@ceph.alertmanager.a.smithi122.stdout:Sep 11 08:16:27 smithi122 ceph-f1d75850-7013-11ef-bcea-c7b262605968-alertmanager-a[73957]: ts=2024-09-11T08:16:27.923Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-09-11T08:16:28.207 INFO:journalctl@ceph.alertmanager.a.smithi122.stdout:Sep 11 08:16:27 smithi122 podman[83461]: 2024-09-11 08:16:27.943984604 +0000 UTC m=+0.117809614 container died bb573119c40ea6a5709763a232d56434b0fc14318b965be0a5f85aae27fb04ae (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T08:16:28.484 INFO:journalctl@ceph.alertmanager.a.smithi122.stdout:Sep 11 08:16:28 smithi122 podman[83461]: 2024-09-11 08:16:28.30643097 +0000 UTC m=+0.480255979 container cleanup bb573119c40ea6a5709763a232d56434b0fc14318b965be0a5f85aae27fb04ae (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T08:16:28.484 INFO:journalctl@ceph.alertmanager.a.smithi122.stdout:Sep 11 08:16:28 smithi122 bash[83461]: ceph-f1d75850-7013-11ef-bcea-c7b262605968-alertmanager-a 2024-09-11T08:16:28.969 INFO:journalctl@ceph.alertmanager.a.smithi122.stdout:Sep 11 08:16:28 smithi122 podman[83475]: 2024-09-11 08:16:28.566115323 +0000 UTC m=+0.620465270 container remove bb573119c40ea6a5709763a232d56434b0fc14318b965be0a5f85aae27fb04ae (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T08:16:28.969 INFO:journalctl@ceph.alertmanager.a.smithi122.stdout:Sep 11 08:16:28 smithi122 podman[83475]: 2024-09-11 08:16:28.641549892 +0000 UTC m=+0.695899794 volume remove 4e806b6924b6173ff4daa3c7ad4ac858a68c739540b5e97ff8b59a165fb78a88 2024-09-11T08:16:28.969 INFO:journalctl@ceph.alertmanager.a.smithi122.stdout:Sep 11 08:16:28 smithi122 systemd[1]: ceph-f1d75850-7013-11ef-bcea-c7b262605968@alertmanager.a.service: Deactivated successfully. 2024-09-11T08:16:28.970 INFO:journalctl@ceph.alertmanager.a.smithi122.stdout:Sep 11 08:16:28 smithi122 systemd[1]: Stopped Ceph alertmanager.a for f1d75850-7013-11ef-bcea-c7b262605968. 2024-09-11T08:16:28.970 INFO:journalctl@ceph.alertmanager.a.smithi122.stdout:Sep 11 08:16:28 smithi122 systemd[1]: ceph-f1d75850-7013-11ef-bcea-c7b262605968@alertmanager.a.service: Consumed 1.112s CPU time. 2024-09-11T08:16:29.234 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Sep 11 08:16:28 smithi122 systemd[1]: Stopping Ceph node-exporter.a for f1d75850-7013-11ef-bcea-c7b262605968... 2024-09-11T08:16:29.234 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Sep 11 08:16:29 smithi122 podman[83580]: 2024-09-11 08:16:29.19686974 +0000 UTC m=+0.089705128 container died f25b60c5acc6bffcf0683390d9d9726c724df1582a7c4d3fc840425271e6db01 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-11T08:16:29.733 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Sep 11 08:16:29 smithi122 podman[83580]: 2024-09-11 08:16:29.553086687 +0000 UTC m=+0.445922076 container cleanup f25b60c5acc6bffcf0683390d9d9726c724df1582a7c4d3fc840425271e6db01 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-11T08:16:29.734 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Sep 11 08:16:29 smithi122 bash[83580]: ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-a 2024-09-11T08:16:30.056 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Sep 11 08:16:29 smithi122 podman[83592]: 2024-09-11 08:16:29.779372421 +0000 UTC m=+0.586293266 container remove f25b60c5acc6bffcf0683390d9d9726c724df1582a7c4d3fc840425271e6db01 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-11T08:16:30.056 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Sep 11 08:16:29 smithi122 systemd[1]: ceph-f1d75850-7013-11ef-bcea-c7b262605968@node-exporter.a.service: Main process exited, code=exited, status=143/n/a 2024-09-11T08:16:30.056 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Sep 11 08:16:29 smithi122 systemd[1]: ceph-f1d75850-7013-11ef-bcea-c7b262605968@node-exporter.a.service: Failed with result 'exit-code'. 2024-09-11T08:16:30.056 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Sep 11 08:16:29 smithi122 systemd[1]: Stopped Ceph node-exporter.a for f1d75850-7013-11ef-bcea-c7b262605968. 2024-09-11T08:16:30.056 INFO:journalctl@ceph.node-exporter.a.smithi122.stdout:Sep 11 08:16:29 smithi122 systemd[1]: ceph-f1d75850-7013-11ef-bcea-c7b262605968@node-exporter.a.service: Consumed 2.388s CPU time. 2024-09-11T08:16:30.535 DEBUG:teuthology.orchestra.run.smithi135:> sudo cephadm rm-cluster --fsid f1d75850-7013-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-11T08:16:32.228 INFO:journalctl@ceph.iscsi.iscsi.a.smithi135.stdout:Sep 11 08:16:31 smithi135 systemd[1]: Stopping Ceph iscsi.iscsi.a for f1d75850-7013-11ef-bcea-c7b262605968... 2024-09-11T08:16:32.228 INFO:journalctl@ceph.iscsi.iscsi.a.smithi135.stdout:Sep 11 08:16:32 smithi135 ceph-f1d75850-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[59799]: debug Shutdown received 2024-09-11T08:16:42.430 INFO:journalctl@ceph.iscsi.iscsi.a.smithi135.stdout:Sep 11 08:16:42 smithi135 bash[71309]: time="2024-09-11T08:16:42Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-f1d75850-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a in 10 seconds, resorting to SIGKILL" 2024-09-11T08:16:42.430 INFO:journalctl@ceph.iscsi.iscsi.a.smithi135.stdout:Sep 11 08:16:42 smithi135 podman[71309]: 2024-09-11 08:16:42.129445854 +0000 UTC m=+10.120589687 container died ff21bbb86657bdc71dd8d1591124429e7b892537148a5029545a1c52dcb6e24b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True) 2024-09-11T08:16:42.728 INFO:journalctl@ceph.iscsi.iscsi.a.smithi135.stdout:Sep 11 08:16:42 smithi135 podman[71309]: 2024-09-11 08:16:42.528122972 +0000 UTC m=+10.519266805 container cleanup ff21bbb86657bdc71dd8d1591124429e7b892537148a5029545a1c52dcb6e24b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T08:16:42.728 INFO:journalctl@ceph.iscsi.iscsi.a.smithi135.stdout:Sep 11 08:16:42 smithi135 bash[71309]: ceph-f1d75850-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a 2024-09-11T08:16:43.012 INFO:journalctl@ceph.iscsi.iscsi.a.smithi135.stdout:Sep 11 08:16:42 smithi135 podman[71323]: 2024-09-11 08:16:42.757205192 +0000 UTC m=+0.623457292 container remove ff21bbb86657bdc71dd8d1591124429e7b892537148a5029545a1c52dcb6e24b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_BRANCH=HEAD) 2024-09-11T08:16:43.013 INFO:journalctl@ceph.iscsi.iscsi.a.smithi135.stdout:Sep 11 08:16:42 smithi135 systemd[1]: ceph-f1d75850-7013-11ef-bcea-c7b262605968@iscsi.iscsi.a.service: Main process exited, code=exited, status=137/n/a 2024-09-11T08:16:43.344 INFO:journalctl@ceph.iscsi.iscsi.a.smithi135.stdout:Sep 11 08:16:43 smithi135 podman[71374]: 2024-09-11 08:16:43.012702037 +0000 UTC m=+0.121238710 container died 004c0dd24ec1098cbefae17ab5bab28ab4bc593b0ff44272fad4e38e58eae639 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-11T08:16:43.621 INFO:journalctl@ceph.iscsi.iscsi.a.smithi135.stdout:Sep 11 08:16:43 smithi135 podman[71374]: 2024-09-11 08:16:43.426534688 +0000 UTC m=+0.535071357 container cleanup 004c0dd24ec1098cbefae17ab5bab28ab4bc593b0ff44272fad4e38e58eae639 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-11T08:16:43.622 INFO:journalctl@ceph.iscsi.iscsi.a.smithi135.stdout:Sep 11 08:16:43 smithi135 bash[71374]: ceph-f1d75850-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu 2024-09-11T08:16:43.622 INFO:journalctl@ceph.iscsi.iscsi.a.smithi135.stdout:Sep 11 08:16:43 smithi135 systemd[1]: ceph-f1d75850-7013-11ef-bcea-c7b262605968@iscsi.iscsi.a.service: Failed with result 'exit-code'. 2024-09-11T08:16:43.622 INFO:journalctl@ceph.iscsi.iscsi.a.smithi135.stdout:Sep 11 08:16:43 smithi135 systemd[1]: ceph-f1d75850-7013-11ef-bcea-c7b262605968@iscsi.iscsi.a.service: Unit process 59688 (conmon) remains running after unit stopped. 2024-09-11T08:16:43.622 INFO:journalctl@ceph.iscsi.iscsi.a.smithi135.stdout:Sep 11 08:16:43 smithi135 systemd[1]: ceph-f1d75850-7013-11ef-bcea-c7b262605968@iscsi.iscsi.a.service: Unit process 71386 (podman) remains running after unit stopped. 2024-09-11T08:16:43.622 INFO:journalctl@ceph.iscsi.iscsi.a.smithi135.stdout:Sep 11 08:16:43 smithi135 systemd[1]: Stopped Ceph iscsi.iscsi.a for f1d75850-7013-11ef-bcea-c7b262605968. 2024-09-11T08:16:43.622 INFO:journalctl@ceph.iscsi.iscsi.a.smithi135.stdout:Sep 11 08:16:43 smithi135 systemd[1]: ceph-f1d75850-7013-11ef-bcea-c7b262605968@iscsi.iscsi.a.service: Consumed 2.648s CPU time. 2024-09-11T08:16:43.978 INFO:journalctl@ceph.iscsi.iscsi.a.smithi135.stdout:Sep 11 08:16:43 smithi135 podman[71386]: 2024-09-11 08:16:43.672840335 +0000 UTC m=+0.663252468 container remove 004c0dd24ec1098cbefae17ab5bab28ab4bc593b0ff44272fad4e38e58eae639 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-11T08:16:43.978 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:16:43 smithi135 systemd[1]: Stopping Ceph grafana.a for f1d75850-7013-11ef-bcea-c7b262605968... 2024-09-11T08:16:44.381 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:16:44 smithi135 podman[71508]: 2024-09-11 08:16:44.075194935 +0000 UTC m=+0.094384679 container died 3d910835e226b370d305848077dbaa374ad787eb07a96ff345b4ee31f93aeaaa (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a, name=ubi8, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, description=Ceph Grafana Container, io.openshift.tags=base rhel8, release=239.1651231664, io.buildah.version=1.32.0, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., summary=Grafana Container configured for Ceph mgr/dashboard integration, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, architecture=x86_64, com.redhat.component=ubi8-container, version=8.5, build-date=2022-04-29T11:48:11.055897, io.openshift.expose-services=, vcs-type=git, vendor=Red Hat, Inc., vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, io.k8s.display-name=Red Hat Universal Base Image 8, distribution-scope=public, maintainer=Paul Cuzner ) 2024-09-11T08:16:44.381 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:16:44 smithi135 podman[71508]: 2024-09-11 08:16:44.380801323 +0000 UTC m=+0.399991073 container cleanup 3d910835e226b370d305848077dbaa374ad787eb07a96ff345b4ee31f93aeaaa (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a, io.k8s.display-name=Red Hat Universal Base Image 8, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, name=ubi8, summary=Grafana Container configured for Ceph mgr/dashboard integration, architecture=x86_64, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, version=8.5, build-date=2022-04-29T11:48:11.055897, io.buildah.version=1.32.0, io.openshift.expose-services=, com.redhat.component=ubi8-container, description=Ceph Grafana Container, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, maintainer=Paul Cuzner , vcs-type=git, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., io.openshift.tags=base rhel8, release=239.1651231664, distribution-scope=public, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, vendor=Red Hat, Inc.) 2024-09-11T08:16:44.689 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:16:44 smithi135 bash[71508]: ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a 2024-09-11T08:16:44.689 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:16:44 smithi135 podman[71520]: 2024-09-11 08:16:44.565364095 +0000 UTC m=+0.488280068 container remove 3d910835e226b370d305848077dbaa374ad787eb07a96ff345b4ee31f93aeaaa (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-grafana-a, vendor=Red Hat, Inc., com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, maintainer=Paul Cuzner , vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, summary=Grafana Container configured for Ceph mgr/dashboard integration, version=8.5, io.openshift.expose-services=, description=Ceph Grafana Container, io.buildah.version=1.32.0, architecture=x86_64, build-date=2022-04-29T11:48:11.055897, name=ubi8, io.k8s.display-name=Red Hat Universal Base Image 8, vcs-type=git, com.redhat.component=ubi8-container, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., io.openshift.tags=base rhel8, release=239.1651231664, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, distribution-scope=public) 2024-09-11T08:16:44.690 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:16:44 smithi135 systemd[1]: ceph-f1d75850-7013-11ef-bcea-c7b262605968@grafana.a.service: Main process exited, code=exited, status=143/n/a 2024-09-11T08:16:44.981 INFO:journalctl@ceph.node-exporter.b.smithi135.stdout:Sep 11 08:16:44 smithi135 systemd[1]: Stopping Ceph node-exporter.b for f1d75850-7013-11ef-bcea-c7b262605968... 2024-09-11T08:16:44.981 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:16:44 smithi135 systemd[1]: ceph-f1d75850-7013-11ef-bcea-c7b262605968@grafana.a.service: Failed with result 'exit-code'. 2024-09-11T08:16:44.981 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:16:44 smithi135 systemd[1]: Stopped Ceph grafana.a for f1d75850-7013-11ef-bcea-c7b262605968. 2024-09-11T08:16:44.981 INFO:journalctl@ceph.grafana.a.smithi135.stdout:Sep 11 08:16:44 smithi135 systemd[1]: ceph-f1d75850-7013-11ef-bcea-c7b262605968@grafana.a.service: Consumed 6.781s CPU time. 2024-09-11T08:16:45.431 INFO:journalctl@ceph.node-exporter.b.smithi135.stdout:Sep 11 08:16:45 smithi135 podman[71628]: 2024-09-11 08:16:45.125561746 +0000 UTC m=+0.095686462 container died 17ab6fbddd24434760d64d36180e166e0509bc08557b2055cb345b6947848323 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-11T08:16:45.431 INFO:journalctl@ceph.node-exporter.b.smithi135.stdout:Sep 11 08:16:45 smithi135 podman[71628]: 2024-09-11 08:16:45.431330364 +0000 UTC m=+0.401455075 container cleanup 17ab6fbddd24434760d64d36180e166e0509bc08557b2055cb345b6947848323 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-11T08:16:45.727 INFO:journalctl@ceph.node-exporter.b.smithi135.stdout:Sep 11 08:16:45 smithi135 bash[71628]: ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-b 2024-09-11T08:16:45.728 INFO:journalctl@ceph.node-exporter.b.smithi135.stdout:Sep 11 08:16:45 smithi135 podman[71641]: 2024-09-11 08:16:45.649760181 +0000 UTC m=+0.527910487 container remove 17ab6fbddd24434760d64d36180e166e0509bc08557b2055cb345b6947848323 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-f1d75850-7013-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-11T08:16:45.728 INFO:journalctl@ceph.node-exporter.b.smithi135.stdout:Sep 11 08:16:45 smithi135 systemd[1]: ceph-f1d75850-7013-11ef-bcea-c7b262605968@node-exporter.b.service: Main process exited, code=exited, status=143/n/a 2024-09-11T08:16:46.098 INFO:journalctl@ceph.node-exporter.b.smithi135.stdout:Sep 11 08:16:45 smithi135 systemd[1]: ceph-f1d75850-7013-11ef-bcea-c7b262605968@node-exporter.b.service: Failed with result 'exit-code'. 2024-09-11T08:16:46.098 INFO:journalctl@ceph.node-exporter.b.smithi135.stdout:Sep 11 08:16:45 smithi135 systemd[1]: Stopped Ceph node-exporter.b for f1d75850-7013-11ef-bcea-c7b262605968. 2024-09-11T08:16:46.098 INFO:journalctl@ceph.node-exporter.b.smithi135.stdout:Sep 11 08:16:45 smithi135 systemd[1]: ceph-f1d75850-7013-11ef-bcea-c7b262605968@node-exporter.b.service: Consumed 2.436s CPU time. 2024-09-11T08:16:46.412 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T08:16:46.444 DEBUG:teuthology.orchestra.run.smithi135:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T08:16:46.472 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-11T08:16:46.474 DEBUG:teuthology.misc:Transferring archived files from smithi122:/var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi/7901251/remote/smithi122/crash 2024-09-11T08:16:46.475 DEBUG:teuthology.orchestra.run.smithi122:> sudo tar c -f - -C /var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/crash -- . 2024-09-11T08:16:46.518 INFO:teuthology.orchestra.run.smithi122.stderr:tar: /var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-11T08:16:46.518 INFO:teuthology.orchestra.run.smithi122.stderr:tar: Error is not recoverable: exiting now 2024-09-11T08:16:46.521 DEBUG:teuthology.misc:Transferring archived files from smithi135:/var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi/7901251/remote/smithi135/crash 2024-09-11T08:16:46.522 DEBUG:teuthology.orchestra.run.smithi135:> sudo tar c -f - -C /var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/crash -- . 2024-09-11T08:16:46.553 INFO:teuthology.orchestra.run.smithi135.stderr:tar: /var/lib/ceph/f1d75850-7013-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-11T08:16:46.553 INFO:teuthology.orchestra.run.smithi135.stderr:tar: Error is not recoverable: exiting now 2024-09-11T08:16:46.554 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-11T08:16:46.555 DEBUG:teuthology.orchestra.run.smithi122:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/f1d75850-7013-11ef-bcea-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | head -n 1 2024-09-11T08:16:46.586 INFO:teuthology.orchestra.run.smithi122.stderr:grep: /var/log/ceph/f1d75850-7013-11ef-bcea-c7b262605968/ceph.log: No such file or directory 2024-09-11T08:16:46.588 INFO:tasks.cephadm:Compressing logs... 2024-09-11T08:16:46.588 DEBUG:teuthology.orchestra.run.smithi122:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-11T08:16:46.630 DEBUG:teuthology.orchestra.run.smithi135:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-11T08:16:46.655 INFO:teuthology.orchestra.run.smithi122.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/f1d75850-7013-11ef-bcea-c7b262605968/ceph-osd.0.log 2024-09-11T08:16:46.655 INFO:teuthology.orchestra.run.smithi122.stderr:: No such file or directory 2024-09-11T08:16:46.655 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/f1d75850-7013-11ef-bcea-c7b262605968/ceph-osd.1.log 2024-09-11T08:16:46.655 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/f1d75850-7013-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-11T08:16:46.655 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/f1d75850-7013-11ef-bcea-c7b262605968/ceph-osd.0.log: /var/log/ceph/f1d75850-7013-11ef-bcea-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/f1d75850-7013-11ef-bcea-c7b262605968/ceph-mon.c.log 2024-09-11T08:16:46.655 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/f1d75850-7013-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/f1d75850-7013-11ef-bcea-c7b262605968/ceph-mon.a.log 2024-09-11T08:16:46.655 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/f1d75850-7013-11ef-bcea-c7b262605968/ceph-osd.2.log 2024-09-11T08:16:46.657 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/f1d75850-7013-11ef-bcea-c7b262605968/ceph-mon.c.log: /var/log/ceph/f1d75850-7013-11ef-bcea-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/f1d75850-7013-11ef-bcea-c7b262605968/ceph-mgr.y.log 2024-09-11T08:16:46.657 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/f1d75850-7013-11ef-bcea-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/f1d75850-7013-11ef-bcea-c7b262605968/ceph-client.rgw.foo.a.log 2024-09-11T08:16:46.657 INFO:teuthology.orchestra.run.smithi135.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/f1d75850-7013-11ef-bcea-c7b262605968/tcmu-runner.log 2024-09-11T08:16:46.658 INFO:teuthology.orchestra.run.smithi135.stderr:: No such file or directory 2024-09-11T08:16:46.658 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/f1d75850-7013-11ef-bcea-c7b262605968/ceph-osd.7.log 2024-09-11T08:16:46.658 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/f1d75850-7013-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-11T08:16:46.658 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/f1d75850-7013-11ef-bcea-c7b262605968/tcmu-runner.log: 63.3% -- replaced with /var/log/ceph/f1d75850-7013-11ef-bcea-c7b262605968/tcmu-runner.log.gz 2024-09-11T08:16:46.658 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/f1d75850-7013-11ef-bcea-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/f1d75850-7013-11ef-bcea-c7b262605968/ceph-osd.4.log 2024-09-11T08:16:46.658 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/f1d75850-7013-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/f1d75850-7013-11ef-bcea-c7b262605968/ceph-osd.6.log 2024-09-11T08:16:46.659 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/f1d75850-7013-11ef-bcea-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/f1d75850-7013-11ef-bcea-c7b262605968/ceph-osd.3.log 2024-09-11T08:16:46.659 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/f1d75850-7013-11ef-bcea-c7b262605968/ceph-client.rgw.foo.a.log: 58.7% -- replaced with /var/log/ceph/f1d75850-7013-11ef-bcea-c7b262605968/ceph-client.rgw.foo.a.log.gz 2024-09-11T08:16:46.659 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-11T08:16:46.661 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/f1d75850-7013-11ef-bcea-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/f1d75850-7013-11ef-bcea-c7b262605968/ceph-osd.5.log 2024-09-11T08:16:46.661 INFO:teuthology.orchestra.run.smithi135.stderr:gzip/var/log/ceph/f1d75850-7013-11ef-bcea-c7b262605968/ceph-osd.6.log: -5 --verbose -- /var/log/ceph/f1d75850-7013-11ef-bcea-c7b262605968/ceph-mgr.x.log 2024-09-11T08:16:46.661 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/f1d75850-7013-11ef-bcea-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/f1d75850-7013-11ef-bcea-c7b262605968/ceph-mon.b.log 2024-09-11T08:16:46.661 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/f1d75850-7013-11ef-bcea-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-11T08:16:46.661 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/f1d75850-7013-11ef-bcea-c7b262605968/ceph-mon.b.log: /var/log/ceph/cephadm.log: 92.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-11T08:16:46.666 INFO:teuthology.orchestra.run.smithi135.stderr: 91.1% -- replaced with /var/log/ceph/f1d75850-7013-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-11T08:16:46.667 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/f1d75850-7013-11ef-bcea-c7b262605968/ceph-osd.3.log: 91.1% -- replaced with /var/log/ceph/f1d75850-7013-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-11T08:16:46.672 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/cephadm.log: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-11T08:16:46.677 INFO:teuthology.orchestra.run.smithi135.stderr: 89.0% -- replaced with /var/log/ceph/f1d75850-7013-11ef-bcea-c7b262605968/ceph-mgr.x.log.gz 2024-09-11T08:16:46.803 INFO:teuthology.orchestra.run.smithi122.stderr: 89.0% -- replaced with /var/log/ceph/f1d75850-7013-11ef-bcea-c7b262605968/ceph-mgr.y.log.gz 2024-09-11T08:16:46.898 INFO:teuthology.orchestra.run.smithi135.stderr: 92.2% -- replaced with /var/log/ceph/f1d75850-7013-11ef-bcea-c7b262605968/ceph-mon.b.log.gz 2024-09-11T08:16:46.947 INFO:teuthology.orchestra.run.smithi122.stderr: 92.0% -- replaced with /var/log/ceph/f1d75850-7013-11ef-bcea-c7b262605968/ceph-mon.c.log.gz 2024-09-11T08:16:46.973 INFO:teuthology.orchestra.run.smithi135.stderr: 92.9% -- replaced with /var/log/ceph/f1d75850-7013-11ef-bcea-c7b262605968/ceph-osd.6.log.gz 2024-09-11T08:16:47.007 INFO:teuthology.orchestra.run.smithi135.stderr: 93.1% -- replaced with /var/log/ceph/f1d75850-7013-11ef-bcea-c7b262605968/ceph-osd.5.log.gz 2024-09-11T08:16:47.016 INFO:teuthology.orchestra.run.smithi122.stderr: 92.8% -- replaced with /var/log/ceph/f1d75850-7013-11ef-bcea-c7b262605968/ceph-osd.2.log.gz 2024-09-11T08:16:47.025 INFO:teuthology.orchestra.run.smithi135.stderr: 93.2% -- replaced with /var/log/ceph/f1d75850-7013-11ef-bcea-c7b262605968/ceph-osd.4.log.gz 2024-09-11T08:16:47.081 INFO:teuthology.orchestra.run.smithi135.stderr: 93.2% -- replaced with /var/log/ceph/f1d75850-7013-11ef-bcea-c7b262605968/ceph-osd.7.log.gz 2024-09-11T08:16:47.082 INFO:teuthology.orchestra.run.smithi135.stderr: 2024-09-11T08:16:47.082 INFO:teuthology.orchestra.run.smithi135.stderr:real 0m0.438s 2024-09-11T08:16:47.082 INFO:teuthology.orchestra.run.smithi135.stderr:user 0m1.680s 2024-09-11T08:16:47.082 INFO:teuthology.orchestra.run.smithi135.stderr:sys 0m0.083s 2024-09-11T08:16:47.086 INFO:teuthology.orchestra.run.smithi122.stderr: 92.9% -- replaced with /var/log/ceph/f1d75850-7013-11ef-bcea-c7b262605968/ceph-osd.0.log.gz 2024-09-11T08:16:47.181 INFO:teuthology.orchestra.run.smithi122.stderr: 93.2% -- replaced with /var/log/ceph/f1d75850-7013-11ef-bcea-c7b262605968/ceph-osd.3.log.gz 2024-09-11T08:16:47.207 INFO:teuthology.orchestra.run.smithi122.stderr: 91.1% -- replaced with /var/log/ceph/f1d75850-7013-11ef-bcea-c7b262605968/ceph-mon.a.log.gz 2024-09-11T08:16:47.210 INFO:teuthology.orchestra.run.smithi122.stderr: 93.1% -- replaced with /var/log/ceph/f1d75850-7013-11ef-bcea-c7b262605968/ceph-osd.1.log.gz 2024-09-11T08:16:47.212 INFO:teuthology.orchestra.run.smithi122.stderr: 2024-09-11T08:16:47.212 INFO:teuthology.orchestra.run.smithi122.stderr:real 0m0.569s 2024-09-11T08:16:47.212 INFO:teuthology.orchestra.run.smithi122.stderr:user 0m2.787s 2024-09-11T08:16:47.212 INFO:teuthology.orchestra.run.smithi122.stderr:sys 0m0.116s 2024-09-11T08:16:47.212 INFO:tasks.cephadm:Archiving logs... 2024-09-11T08:16:47.213 DEBUG:teuthology.misc:Transferring archived files from smithi122:/var/log/ceph to /home/teuthworker/archive/gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi/7901251/remote/smithi122/log 2024-09-11T08:16:47.213 DEBUG:teuthology.orchestra.run.smithi122:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-11T08:16:47.651 DEBUG:teuthology.misc:Transferring archived files from smithi135:/var/log/ceph to /home/teuthworker/archive/gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi/7901251/remote/smithi135/log 2024-09-11T08:16:47.652 DEBUG:teuthology.orchestra.run.smithi135:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-11T08:16:47.916 INFO:tasks.cephadm:Removing cluster... 2024-09-11T08:16:47.916 DEBUG:teuthology.orchestra.run.smithi122:> sudo cephadm rm-cluster --fsid f1d75850-7013-11ef-bcea-c7b262605968 --force 2024-09-11T08:16:48.394 DEBUG:teuthology.orchestra.run.smithi135:> sudo cephadm rm-cluster --fsid f1d75850-7013-11ef-bcea-c7b262605968 --force 2024-09-11T08:16:48.859 INFO:tasks.cephadm:Teardown complete 2024-09-11T08:16:48.859 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-11T08:16:48.869 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-11T08:16:48.869 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-11T08:16:48.872 DEBUG:teuthology.orchestra.run.smithi135:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-11T08:16:48.951 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-11T08:16:49.031 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-11T08:16:49.031 DEBUG:teuthology.orchestra.run.smithi122:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T08:16:49.033 DEBUG:teuthology.orchestra.run.smithi135:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T08:16:49.046 INFO:teuthology.orchestra.run.smithi122.stderr:bash: line 1: ntpq: command not found 2024-09-11T08:16:49.048 INFO:teuthology.orchestra.run.smithi135.stderr:bash: line 1: ntpq: command not found 2024-09-11T08:16:49.050 INFO:teuthology.orchestra.run.smithi122.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T08:16:49.051 INFO:teuthology.orchestra.run.smithi122.stdout:=============================================================================== 2024-09-11T08:16:49.051 INFO:teuthology.orchestra.run.smithi122.stdout:^+ hv01.front.sepia.ceph.com 3 6 377 13 -1424us[-1395us] +/- 75ms 2024-09-11T08:16:49.051 INFO:teuthology.orchestra.run.smithi122.stdout:^* hv02.front.sepia.ceph.com 3 6 377 12 +771us[ +800us] +/- 51ms 2024-09-11T08:16:49.051 INFO:teuthology.orchestra.run.smithi122.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 10 +289us[ +289us] +/- 66ms 2024-09-11T08:16:49.051 INFO:teuthology.orchestra.run.smithi122.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T08:16:49.052 INFO:teuthology.orchestra.run.smithi135.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T08:16:49.052 INFO:teuthology.orchestra.run.smithi135.stdout:=============================================================================== 2024-09-11T08:16:49.052 INFO:teuthology.orchestra.run.smithi135.stdout:^+ hv01.front.sepia.ceph.com 3 6 377 12 -1414us[-1412us] +/- 75ms 2024-09-11T08:16:49.052 INFO:teuthology.orchestra.run.smithi135.stdout:^* hv02.front.sepia.ceph.com 3 6 377 8 +774us[ +776us] +/- 51ms 2024-09-11T08:16:49.052 INFO:teuthology.orchestra.run.smithi135.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 12 +294us[ +295us] +/- 66ms 2024-09-11T08:16:49.052 INFO:teuthology.orchestra.run.smithi135.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T08:16:49.053 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-11T08:16:49.155 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-11T08:16:49.155 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-11T08:16:49.174 DEBUG:teuthology.orchestra.run.smithi122:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-11T08:16:49.248 DEBUG:teuthology.orchestra.run.smithi135:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-11T08:16:49.319 DEBUG:teuthology.orchestra.run.smithi122:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-11T08:16:49.346 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:16:49.347 DEBUG:teuthology.orchestra.run.smithi135:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-11T08:16:49.375 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:16:49.375 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-11T08:16:49.386 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-11T08:16:49.398 INFO:teuthology.task.internal:Duration was 1340.116326 seconds 2024-09-11T08:16:49.399 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-11T08:16:49.491 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-11T08:16:49.491 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-11T08:16:49.494 DEBUG:teuthology.orchestra.run.smithi135:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-11T08:16:49.530 INFO:teuthology.orchestra.run.smithi122.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T08:16:49.532 INFO:teuthology.orchestra.run.smithi135.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T08:16:50.021 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-11T08:16:50.021 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi122.front.sepia.ceph.com 2024-09-11T08:16:50.022 DEBUG:teuthology.orchestra.run.smithi122:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-11T08:16:50.084 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi135.front.sepia.ceph.com 2024-09-11T08:16:50.084 DEBUG:teuthology.orchestra.run.smithi135:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-11T08:16:50.113 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-11T08:16:50.113 DEBUG:teuthology.orchestra.run.smithi122:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-11T08:16:50.127 DEBUG:teuthology.orchestra.run.smithi135:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-11T08:16:50.181 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-11T08:16:50.181 DEBUG:teuthology.orchestra.run.smithi122:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-11T08:16:50.193 DEBUG:teuthology.orchestra.run.smithi135:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-11T08:16:50.469 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-11T08:16:50.479 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-11T08:16:50.480 DEBUG:teuthology.orchestra.run.smithi122:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-11T08:16:50.505 DEBUG:teuthology.orchestra.run.smithi135:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-11T08:16:50.530 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-11T08:16:50.540 DEBUG:teuthology.orchestra.run.smithi122:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-11T08:16:50.547 DEBUG:teuthology.orchestra.run.smithi135:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-11T08:16:50.569 INFO:teuthology.orchestra.run.smithi122.stdout:kernel.core_pattern = core 2024-09-11T08:16:50.594 INFO:teuthology.orchestra.run.smithi135.stdout:kernel.core_pattern = core 2024-09-11T08:16:50.648 DEBUG:teuthology.orchestra.run.smithi122:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-11T08:16:50.662 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:16:50.663 DEBUG:teuthology.orchestra.run.smithi135:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-11T08:16:50.703 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:16:50.703 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-11T08:16:50.715 INFO:teuthology.task.internal:Transferring archived files... 2024-09-11T08:16:50.716 DEBUG:teuthology.misc:Transferring archived files from smithi122:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi/7901251/remote/smithi122 2024-09-11T08:16:50.717 DEBUG:teuthology.orchestra.run.smithi122:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-11T08:16:50.758 DEBUG:teuthology.misc:Transferring archived files from smithi135:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi/7901251/remote/smithi135 2024-09-11T08:16:50.758 DEBUG:teuthology.orchestra.run.smithi135:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-11T08:16:50.794 INFO:teuthology.task.internal:Removing archive directory... 2024-09-11T08:16:50.795 DEBUG:teuthology.orchestra.run.smithi122:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-11T08:16:50.797 DEBUG:teuthology.orchestra.run.smithi135:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-11T08:16:50.844 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-11T08:16:50.855 INFO:teuthology.task.internal:Not uploading archives. 2024-09-11T08:16:50.855 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-11T08:16:50.907 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-11T08:16:50.907 DEBUG:teuthology.orchestra.run.smithi122:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-11T08:16:50.910 DEBUG:teuthology.orchestra.run.smithi135:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-11T08:16:50.922 INFO:teuthology.orchestra.run.smithi122.stdout: 265336 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 11 08:16 /home/ubuntu/cephtest 2024-09-11T08:16:50.925 INFO:teuthology.orchestra.run.smithi135.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 11 08:16 /home/ubuntu/cephtest 2024-09-11T08:16:50.926 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-11T08:16:50.936 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-11T08:16:50.968 INFO:teuthology.run:Summary data: description: orch:cephadm/with-work/{0-distro/centos_9.stream fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rotate-keys} duration: 1340.1163260936737 flavor: default owner: scheduled_gabrioux@teuthology success: true 2024-09-11T08:16:50.969 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-11T08:16:51.047 INFO:teuthology.run:pass