2024-09-10T18:42:23.549 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-10T18:42:23.564 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-10T18:42:23.654 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/gabrioux-2024-09-10_17:53:37-orch:cephadm-wip-guits-quincy-2024-09-09-1810-quincy-distro-default-smithi/7900270 branch: wip-guits-quincy-2024-09-09-1810-quincy description: orch:cephadm/with-work/{0-distro/centos_9.stream fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rotate-keys} email: gabrioux@ibm.com first_in_suite: false job_id: '7900270' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: gabrioux-2024-09-10_17:53:37-orch:cephadm-wip-guits-quincy-2024-09-09-1810-quincy-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-guits-quincy-2024-09-09-1810-quincy ceph: conf: global: mon election default strategy: 3 ms bind msgr1: false ms bind msgr2: true ms type: async mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: debug ms: 1 debug osd: 20 osd shutdown pgref assert: true flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) sha1: 5be57395ea235535910512ae73ba8e7ff25fe588 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} cephadm: cephadm_mode: root install: ceph: flavor: default sha1: 5be57395ea235535910512ae73ba8e7ff25fe588 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 workunit: branch: quincy-orch-centos-9-teuth sha1: fad5356a94e6044ce1062d748daa5a0f2ed53a7d owner: scheduled_gabrioux@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - ceph.rgw.foo.a - node-exporter.a - alertmanager.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - grafana.a - node-exporter.b - ceph.iscsi.iscsi.a seed: 4625 sha1: 5be57395ea235535910512ae73ba8e7ff25fe588 sleep_before_teardown: 0 subset: 1/8 suite: orch:cephadm suite_branch: quincy-orch-centos-9-teuth suite_path: /home/teuthworker/src/github.com_adk3798_ceph_fad5356a94e6044ce1062d748daa5a0f2ed53a7d/qa suite_relpath: qa suite_repo: https://github.com/adk3798/ceph.git suite_sha1: fad5356a94e6044ce1062d748daa5a0f2ed53a7d targets: smithi047.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGGTYx81sHUcacayoNSMR5+G5nP4NgNE0Jg52IwVrcq42UnRPCIRD3Y3/QCKJviFZQE5W6675OhFlD0EGdwypzk= smithi149.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJ7OKST7q3ruSt5vzitA9m3Y4dJYn85Qzs4+7bkmS4fMCZ0KXCudshedlGZcmNpNaK+VB8jN6esYpzdnlttl83Y= tasks: - pexec: all: - sudo dnf remove nvme-cli -y - sudo dnf install nvmetcli nvme-cli -y - install: null - cephadm: conf: mgr: debug mgr: 20 debug ms: 1 - cephadm.shell: mon.a: - "set -ex\nfor f in osd.0 osd.1 osd.2 osd.3 osd.4 osd.5 osd.6 osd.7 mgr.y mgr.x\n\ do\n echo \"rotating key for $f\"\n K=$(ceph auth get-key $f)\n NK=\"\ $K\"\n ceph orch daemon rotate-key $f\n while [ \"$K\" == \"$NK\" ]; do\n\ \ sleep 5\n NK=$(ceph auth get-key $f)\n done\ndone\n" teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-10_17:53:37 tube: smithi user: gabrioux verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-10T18:42:23.654 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_adk3798_ceph_fad5356a94e6044ce1062d748daa5a0f2ed53a7d/qa; will attempt to use it 2024-09-10T18:42:23.655 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_adk3798_ceph_fad5356a94e6044ce1062d748daa5a0f2ed53a7d/qa/tasks 2024-09-10T18:42:23.655 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-10T18:42:23.657 INFO:teuthology.task.internal:Checking packages... 2024-09-10T18:42:23.678 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '5be57395ea235535910512ae73ba8e7ff25fe588' 2024-09-10T18:42:23.678 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-10T18:42:23.678 INFO:teuthology.packaging:ref: None 2024-09-10T18:42:23.678 INFO:teuthology.packaging:tag: None 2024-09-10T18:42:23.678 INFO:teuthology.packaging:branch: wip-guits-quincy-2024-09-09-1810-quincy 2024-09-10T18:42:23.678 INFO:teuthology.packaging:sha1: 5be57395ea235535910512ae73ba8e7ff25fe588 2024-09-10T18:42:23.679 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-guits-quincy-2024-09-09-1810-quincy 2024-09-10T18:42:23.830 INFO:teuthology.task.internal:Found packages for ceph version 17.2.7-1606.g5be57395 2024-09-10T18:42:23.831 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-10T18:42:23.838 INFO:teuthology.task.internal:no buildpackages task found 2024-09-10T18:42:23.838 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-10T18:42:23.856 INFO:teuthology.task.internal:Saving configuration 2024-09-10T18:42:23.867 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-10T18:42:23.874 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-10T18:42:23.900 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi047.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-09-10_17:53:37-orch:cephadm-wip-guits-quincy-2024-09-09-1810-quincy-distro-default-smithi/7900270', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 18:37:14.784848', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGGTYx81sHUcacayoNSMR5+G5nP4NgNE0Jg52IwVrcq42UnRPCIRD3Y3/QCKJviFZQE5W6675OhFlD0EGdwypzk='} 2024-09-10T18:42:23.923 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi149.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-09-10_17:53:37-orch:cephadm-wip-guits-quincy-2024-09-09-1810-quincy-distro-default-smithi/7900270', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 18:37:14.786791', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJ7OKST7q3ruSt5vzitA9m3Y4dJYn85Qzs4+7bkmS4fMCZ0KXCudshedlGZcmNpNaK+VB8jN6esYpzdnlttl83Y='} 2024-09-10T18:42:23.923 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-10T18:42:23.930 INFO:teuthology.task.internal:roles: ubuntu@smithi047.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'ceph.rgw.foo.a', 'node-exporter.a', 'alertmanager.a'] 2024-09-10T18:42:23.930 INFO:teuthology.task.internal:roles: ubuntu@smithi149.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b', 'ceph.iscsi.iscsi.a'] 2024-09-10T18:42:23.930 INFO:teuthology.run_tasks:Running task console_log... 2024-09-10T18:42:23.990 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f54abd63ac0>, signals=[15]) 2024-09-10T18:42:23.990 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-10T18:42:23.996 INFO:teuthology.task.internal:Opening connections... 2024-09-10T18:42:23.996 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi047.front.sepia.ceph.com 2024-09-10T18:42:23.997 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T18:42:24.076 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi149.front.sepia.ceph.com 2024-09-10T18:42:24.077 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T18:42:24.153 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-10T18:42:24.161 DEBUG:teuthology.orchestra.run.smithi047:> uname -m 2024-09-10T18:42:24.179 INFO:teuthology.orchestra.run.smithi047.stdout:x86_64 2024-09-10T18:42:24.180 DEBUG:teuthology.orchestra.run.smithi047:> cat /etc/os-release 2024-09-10T18:42:24.238 INFO:teuthology.orchestra.run.smithi047.stdout:NAME="CentOS Stream" 2024-09-10T18:42:24.238 INFO:teuthology.orchestra.run.smithi047.stdout:VERSION="9" 2024-09-10T18:42:24.238 INFO:teuthology.orchestra.run.smithi047.stdout:ID="centos" 2024-09-10T18:42:24.238 INFO:teuthology.orchestra.run.smithi047.stdout:ID_LIKE="rhel fedora" 2024-09-10T18:42:24.238 INFO:teuthology.orchestra.run.smithi047.stdout:VERSION_ID="9" 2024-09-10T18:42:24.238 INFO:teuthology.orchestra.run.smithi047.stdout:PLATFORM_ID="platform:el9" 2024-09-10T18:42:24.238 INFO:teuthology.orchestra.run.smithi047.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-10T18:42:24.239 INFO:teuthology.orchestra.run.smithi047.stdout:ANSI_COLOR="0;31" 2024-09-10T18:42:24.239 INFO:teuthology.orchestra.run.smithi047.stdout:LOGO="fedora-logo-icon" 2024-09-10T18:42:24.239 INFO:teuthology.orchestra.run.smithi047.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-10T18:42:24.239 INFO:teuthology.orchestra.run.smithi047.stdout:HOME_URL="https://centos.org/" 2024-09-10T18:42:24.239 INFO:teuthology.orchestra.run.smithi047.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-10T18:42:24.239 INFO:teuthology.orchestra.run.smithi047.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-10T18:42:24.239 INFO:teuthology.orchestra.run.smithi047.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-10T18:42:24.240 INFO:teuthology.lock.ops:Updating smithi047.front.sepia.ceph.com on lock server 2024-09-10T18:42:24.269 DEBUG:teuthology.orchestra.run.smithi149:> uname -m 2024-09-10T18:42:24.287 INFO:teuthology.orchestra.run.smithi149.stdout:x86_64 2024-09-10T18:42:24.287 DEBUG:teuthology.orchestra.run.smithi149:> cat /etc/os-release 2024-09-10T18:42:24.344 INFO:teuthology.orchestra.run.smithi149.stdout:NAME="CentOS Stream" 2024-09-10T18:42:24.344 INFO:teuthology.orchestra.run.smithi149.stdout:VERSION="9" 2024-09-10T18:42:24.344 INFO:teuthology.orchestra.run.smithi149.stdout:ID="centos" 2024-09-10T18:42:24.345 INFO:teuthology.orchestra.run.smithi149.stdout:ID_LIKE="rhel fedora" 2024-09-10T18:42:24.345 INFO:teuthology.orchestra.run.smithi149.stdout:VERSION_ID="9" 2024-09-10T18:42:24.345 INFO:teuthology.orchestra.run.smithi149.stdout:PLATFORM_ID="platform:el9" 2024-09-10T18:42:24.345 INFO:teuthology.orchestra.run.smithi149.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-10T18:42:24.345 INFO:teuthology.orchestra.run.smithi149.stdout:ANSI_COLOR="0;31" 2024-09-10T18:42:24.345 INFO:teuthology.orchestra.run.smithi149.stdout:LOGO="fedora-logo-icon" 2024-09-10T18:42:24.346 INFO:teuthology.orchestra.run.smithi149.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-10T18:42:24.346 INFO:teuthology.orchestra.run.smithi149.stdout:HOME_URL="https://centos.org/" 2024-09-10T18:42:24.346 INFO:teuthology.orchestra.run.smithi149.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-10T18:42:24.346 INFO:teuthology.orchestra.run.smithi149.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-10T18:42:24.346 INFO:teuthology.orchestra.run.smithi149.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-10T18:42:24.346 INFO:teuthology.lock.ops:Updating smithi149.front.sepia.ceph.com on lock server 2024-09-10T18:42:24.369 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-10T18:42:24.377 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-10T18:42:24.382 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-10T18:42:24.382 DEBUG:teuthology.orchestra.run.smithi047:> test '!' -e /home/ubuntu/cephtest 2024-09-10T18:42:24.385 DEBUG:teuthology.orchestra.run.smithi149:> test '!' -e /home/ubuntu/cephtest 2024-09-10T18:42:24.401 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-10T18:42:24.408 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-10T18:42:24.408 DEBUG:teuthology.orchestra.run.smithi047:> test -z $(ls -A /var/lib/ceph) 2024-09-10T18:42:24.443 DEBUG:teuthology.orchestra.run.smithi149:> test -z $(ls -A /var/lib/ceph) 2024-09-10T18:42:24.480 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-10T18:42:24.541 INFO:teuthology.run_tasks:Running task kernel... 2024-09-10T18:42:24.556 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-10T18:42:24.556 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-10T18:42:24.556 DEBUG:teuthology.orchestra.run.smithi047:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T18:42:24.557 DEBUG:teuthology.orchestra.run.smithi149:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T18:42:24.573 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:42:24.573 DEBUG:teuthology.orchestra.run.smithi149:> uname -r 2024-09-10T18:42:24.574 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:42:24.575 DEBUG:teuthology.orchestra.run.smithi047:> uname -r 2024-09-10T18:42:24.631 INFO:teuthology.orchestra.run.smithi149.stdout:5.14.0-503.el9.x86_64 2024-09-10T18:42:24.631 INFO:teuthology.task.kernel:Running kernel on smithi149: 5.14.0-503.el9.x86_64 2024-09-10T18:42:24.632 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum install -y kernel 2024-09-10T18:42:24.633 INFO:teuthology.orchestra.run.smithi047.stdout:5.14.0-503.el9.x86_64 2024-09-10T18:42:24.633 INFO:teuthology.task.kernel:Running kernel on smithi047: 5.14.0-503.el9.x86_64 2024-09-10T18:42:24.633 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum install -y kernel 2024-09-10T18:42:28.496 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS Stream 9 - BaseOS 3.9 MB/s | 8.2 MB 00:02 2024-09-10T18:42:30.599 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - BaseOS 1.9 MB/s | 8.2 MB 00:04 2024-09-10T18:42:32.809 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - AppStream 20 MB/s | 20 MB 00:01 2024-09-10T18:42:36.230 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS Stream 9 - AppStream 3.1 MB/s | 20 MB 00:06 2024-09-10T18:42:40.497 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.5 MB 00:00 2024-09-10T18:42:40.509 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - CRB 1.7 MB/s | 6.5 MB 00:03 2024-09-10T18:42:41.906 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS Stream 9 - Extras packages 89 kB/s | 19 kB 00:00 2024-09-10T18:42:41.926 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - Extras packages 106 kB/s | 19 kB 00:00 2024-09-10T18:42:42.785 INFO:teuthology.orchestra.run.smithi149.stdout:Extra Packages for Enterprise Linux 30 MB/s | 23 MB 00:00 2024-09-10T18:42:42.872 INFO:teuthology.orchestra.run.smithi047.stdout:Extra Packages for Enterprise Linux 28 MB/s | 23 MB 00:00 2024-09-10T18:42:48.382 INFO:teuthology.orchestra.run.smithi149.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-09-10T18:42:48.461 INFO:teuthology.orchestra.run.smithi047.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-09-10T18:42:50.447 INFO:teuthology.orchestra.run.smithi149.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-10T18:42:50.447 INFO:teuthology.orchestra.run.smithi149.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-10T18:42:50.539 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-09-10T18:42:50.544 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-09-10T18:42:50.545 INFO:teuthology.orchestra.run.smithi149.stdout: Package Architecture Version Repository Size 2024-09-10T18:42:50.545 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-09-10T18:42:50.545 INFO:teuthology.orchestra.run.smithi149.stdout:Installing: 2024-09-10T18:42:50.545 INFO:teuthology.orchestra.run.smithi149.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-10T18:42:50.545 INFO:teuthology.orchestra.run.smithi149.stdout:Installing dependencies: 2024-09-10T18:42:50.545 INFO:teuthology.orchestra.run.smithi149.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-10T18:42:50.545 INFO:teuthology.orchestra.run.smithi149.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-10T18:42:50.545 INFO:teuthology.orchestra.run.smithi149.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-10T18:42:50.545 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T18:42:50.545 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction Summary 2024-09-10T18:42:50.545 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-09-10T18:42:50.545 INFO:teuthology.orchestra.run.smithi149.stdout:Install 4 Packages 2024-09-10T18:42:50.545 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T18:42:50.546 INFO:teuthology.orchestra.run.smithi149.stdout:Total download size: 79 M 2024-09-10T18:42:50.546 INFO:teuthology.orchestra.run.smithi149.stdout:Installed size: 126 M 2024-09-10T18:42:50.546 INFO:teuthology.orchestra.run.smithi149.stdout:Downloading Packages: 2024-09-10T18:42:50.567 INFO:teuthology.orchestra.run.smithi047.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-10T18:42:50.567 INFO:teuthology.orchestra.run.smithi047.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-10T18:42:50.642 INFO:teuthology.orchestra.run.smithi149.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 300 kB/s | 23 kB 00:00 2024-09-10T18:42:50.654 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-09-10T18:42:50.672 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-10T18:42:50.672 INFO:teuthology.orchestra.run.smithi047.stdout: Package Architecture Version Repository Size 2024-09-10T18:42:50.672 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-10T18:42:50.672 INFO:teuthology.orchestra.run.smithi047.stdout:Installing: 2024-09-10T18:42:50.672 INFO:teuthology.orchestra.run.smithi047.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-10T18:42:50.672 INFO:teuthology.orchestra.run.smithi047.stdout:Installing dependencies: 2024-09-10T18:42:50.673 INFO:teuthology.orchestra.run.smithi047.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-10T18:42:50.673 INFO:teuthology.orchestra.run.smithi047.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-10T18:42:50.673 INFO:teuthology.orchestra.run.smithi047.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-10T18:42:50.673 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:42:50.673 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction Summary 2024-09-10T18:42:50.673 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-10T18:42:50.673 INFO:teuthology.orchestra.run.smithi047.stdout:Install 4 Packages 2024-09-10T18:42:50.673 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:42:50.674 INFO:teuthology.orchestra.run.smithi047.stdout:Total download size: 79 M 2024-09-10T18:42:50.674 INFO:teuthology.orchestra.run.smithi047.stdout:Installed size: 126 M 2024-09-10T18:42:50.674 INFO:teuthology.orchestra.run.smithi047.stdout:Downloading Packages: 2024-09-10T18:42:50.854 INFO:teuthology.orchestra.run.smithi047.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 272 kB/s | 23 kB 00:00 2024-09-10T18:42:53.546 INFO:teuthology.orchestra.run.smithi047.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 5.6 MB/s | 16 MB 00:02 2024-09-10T18:42:54.608 INFO:teuthology.orchestra.run.smithi149.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 3.9 MB/s | 16 MB 00:04 2024-09-10T18:42:55.846 INFO:teuthology.orchestra.run.smithi047.stdout:(3/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 6.8 MB/s | 35 MB 00:05 2024-09-10T18:42:56.488 INFO:teuthology.orchestra.run.smithi047.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 5.1 MB/s | 29 MB 00:05 2024-09-10T18:42:56.489 INFO:teuthology.orchestra.run.smithi047.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:42:56.489 INFO:teuthology.orchestra.run.smithi047.stdout:Total 14 MB/s | 79 MB 00:05 2024-09-10T18:42:57.202 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction check 2024-09-10T18:42:57.579 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction check succeeded. 2024-09-10T18:42:57.579 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction test 2024-09-10T18:42:58.058 INFO:teuthology.orchestra.run.smithi149.stdout:(3/4): kernel-modules-core-5.14.0-505.el9.x86_6 3.8 MB/s | 29 MB 00:07 2024-09-10T18:42:58.559 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction test succeeded. 2024-09-10T18:42:58.559 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction 2024-09-10T18:42:59.700 INFO:teuthology.orchestra.run.smithi149.stdout:(4/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 3.8 MB/s | 35 MB 00:09 2024-09-10T18:42:59.700 INFO:teuthology.orchestra.run.smithi149.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:42:59.700 INFO:teuthology.orchestra.run.smithi149.stdout:Total 8.6 MB/s | 79 MB 00:09 2024-09-10T18:43:00.384 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction check 2024-09-10T18:43:00.774 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction check succeeded. 2024-09-10T18:43:00.774 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction test 2024-09-10T18:43:01.122 INFO:teuthology.orchestra.run.smithi047.stdout: Preparing : 1/1 2024-09-10T18:43:01.727 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction test succeeded. 2024-09-10T18:43:01.727 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction 2024-09-10T18:43:01.801 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-10T18:43:01.882 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T18:43:04.200 INFO:teuthology.orchestra.run.smithi149.stdout: Preparing : 1/1 2024-09-10T18:43:04.676 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T18:43:04.816 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T18:43:04.849 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-10T18:43:04.942 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T18:43:07.747 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T18:43:07.872 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T18:43:09.947 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T18:43:10.016 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-10T18:43:12.719 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T18:43:12.753 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-10T18:43:15.156 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T18:43:17.589 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T18:43:54.575 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T18:43:54.616 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-10T18:43:54.989 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-10T18:43:54.989 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-10T18:43:54.989 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T18:43:54.989 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T18:43:55.829 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T18:43:55.829 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:43:55.829 INFO:teuthology.orchestra.run.smithi047.stdout:Installed: 2024-09-10T18:43:55.829 INFO:teuthology.orchestra.run.smithi047.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-10T18:43:55.829 INFO:teuthology.orchestra.run.smithi047.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-10T18:43:55.830 INFO:teuthology.orchestra.run.smithi047.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-10T18:43:55.830 INFO:teuthology.orchestra.run.smithi047.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-10T18:43:55.830 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:43:55.830 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-09-10T18:43:55.885 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T18:43:55.926 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-10T18:43:56.221 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-10T18:43:56.221 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-10T18:43:56.221 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T18:43:56.221 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T18:43:56.262 DEBUG:teuthology.orchestra.run.smithi047:> rpm -q kernel | sort -rV | head -n 1 2024-09-10T18:43:56.361 INFO:teuthology.orchestra.run.smithi047.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-10T18:43:56.362 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-10T18:43:56.362 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-10T18:43:56.362 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-10T18:43:56.362 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-10T18:43:56.363 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-10T18:43:56.363 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi047.front.sepia.ceph.com, path=None, version=distro) 2024-09-10T18:43:56.363 DEBUG:teuthology.orchestra.run.smithi047:> rpm -q kernel | sort -rV | head -n 1 2024-09-10T18:43:56.423 INFO:teuthology.orchestra.run.smithi047.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-10T18:43:56.423 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-10T18:43:56.423 DEBUG:teuthology.orchestra.run.smithi047:> sudo rpm -qi grub2-tools 2024-09-10T18:43:56.463 INFO:teuthology.orchestra.run.smithi047.stdout:Name : grub2-tools 2024-09-10T18:43:56.463 INFO:teuthology.orchestra.run.smithi047.stdout:Epoch : 1 2024-09-10T18:43:56.463 INFO:teuthology.orchestra.run.smithi047.stdout:Version : 2.06 2024-09-10T18:43:56.463 INFO:teuthology.orchestra.run.smithi047.stdout:Release : 82.el9 2024-09-10T18:43:56.463 INFO:teuthology.orchestra.run.smithi047.stdout:Architecture: x86_64 2024-09-10T18:43:56.463 INFO:teuthology.orchestra.run.smithi047.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-10T18:43:56.464 INFO:teuthology.orchestra.run.smithi047.stdout:Group : Unspecified 2024-09-10T18:43:56.464 INFO:teuthology.orchestra.run.smithi047.stdout:Size : 8274670 2024-09-10T18:43:56.464 INFO:teuthology.orchestra.run.smithi047.stdout:License : GPLv3+ 2024-09-10T18:43:56.464 INFO:teuthology.orchestra.run.smithi047.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-10T18:43:56.464 INFO:teuthology.orchestra.run.smithi047.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-10T18:43:56.464 INFO:teuthology.orchestra.run.smithi047.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-10T18:43:56.465 INFO:teuthology.orchestra.run.smithi047.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-10T18:43:56.465 INFO:teuthology.orchestra.run.smithi047.stdout:Packager : builder@centos.org 2024-09-10T18:43:56.465 INFO:teuthology.orchestra.run.smithi047.stdout:Vendor : CentOS 2024-09-10T18:43:56.465 INFO:teuthology.orchestra.run.smithi047.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-10T18:43:56.465 INFO:teuthology.orchestra.run.smithi047.stdout:Summary : Support tools for GRUB. 2024-09-10T18:43:56.465 INFO:teuthology.orchestra.run.smithi047.stdout:Description : 2024-09-10T18:43:56.465 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:43:56.465 INFO:teuthology.orchestra.run.smithi047.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-10T18:43:56.465 INFO:teuthology.orchestra.run.smithi047.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-10T18:43:56.465 INFO:teuthology.orchestra.run.smithi047.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-10T18:43:56.465 INFO:teuthology.orchestra.run.smithi047.stdout:hardware devices. 2024-09-10T18:43:56.465 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:43:56.465 INFO:teuthology.orchestra.run.smithi047.stdout:This subpackage provides tools for support of all platforms. 2024-09-10T18:43:56.466 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-10T18:43:56.466 INFO:teuthology.task.kernel:Updating grub on smithi047 to boot 5.14.0-505.el9.x86_64 2024-09-10T18:43:56.467 DEBUG:teuthology.orchestra.run.smithi047:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-10T18:43:57.059 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T18:43:57.059 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T18:43:57.059 INFO:teuthology.orchestra.run.smithi149.stdout:Installed: 2024-09-10T18:43:57.059 INFO:teuthology.orchestra.run.smithi149.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-10T18:43:57.059 INFO:teuthology.orchestra.run.smithi149.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-10T18:43:57.060 INFO:teuthology.orchestra.run.smithi149.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-10T18:43:57.060 INFO:teuthology.orchestra.run.smithi149.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-10T18:43:57.060 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T18:43:57.060 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-09-10T18:43:57.143 INFO:teuthology.orchestra.run.smithi047.stderr:Generating grub configuration file ... 2024-09-10T18:43:57.409 DEBUG:teuthology.orchestra.run.smithi149:> rpm -q kernel | sort -rV | head -n 1 2024-09-10T18:43:57.497 INFO:teuthology.orchestra.run.smithi149.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-10T18:43:57.497 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-10T18:43:57.497 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-10T18:43:57.497 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-10T18:43:57.497 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-10T18:43:57.497 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-10T18:43:57.498 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi149.front.sepia.ceph.com, path=None, version=distro) 2024-09-10T18:43:57.498 DEBUG:teuthology.orchestra.run.smithi149:> rpm -q kernel | sort -rV | head -n 1 2024-09-10T18:43:57.557 INFO:teuthology.orchestra.run.smithi149.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-10T18:43:57.558 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-10T18:43:57.558 DEBUG:teuthology.orchestra.run.smithi149:> sudo rpm -qi grub2-tools 2024-09-10T18:43:57.591 INFO:teuthology.orchestra.run.smithi149.stdout:Name : grub2-tools 2024-09-10T18:43:57.591 INFO:teuthology.orchestra.run.smithi149.stdout:Epoch : 1 2024-09-10T18:43:57.591 INFO:teuthology.orchestra.run.smithi149.stdout:Version : 2.06 2024-09-10T18:43:57.591 INFO:teuthology.orchestra.run.smithi149.stdout:Release : 82.el9 2024-09-10T18:43:57.591 INFO:teuthology.orchestra.run.smithi149.stdout:Architecture: x86_64 2024-09-10T18:43:57.591 INFO:teuthology.orchestra.run.smithi149.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-10T18:43:57.591 INFO:teuthology.orchestra.run.smithi149.stdout:Group : Unspecified 2024-09-10T18:43:57.592 INFO:teuthology.orchestra.run.smithi149.stdout:Size : 8274670 2024-09-10T18:43:57.592 INFO:teuthology.orchestra.run.smithi149.stdout:License : GPLv3+ 2024-09-10T18:43:57.592 INFO:teuthology.orchestra.run.smithi149.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-10T18:43:57.592 INFO:teuthology.orchestra.run.smithi149.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-10T18:43:57.592 INFO:teuthology.orchestra.run.smithi149.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-10T18:43:57.592 INFO:teuthology.orchestra.run.smithi149.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-10T18:43:57.592 INFO:teuthology.orchestra.run.smithi149.stdout:Packager : builder@centos.org 2024-09-10T18:43:57.592 INFO:teuthology.orchestra.run.smithi149.stdout:Vendor : CentOS 2024-09-10T18:43:57.593 INFO:teuthology.orchestra.run.smithi149.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-10T18:43:57.593 INFO:teuthology.orchestra.run.smithi149.stdout:Summary : Support tools for GRUB. 2024-09-10T18:43:57.593 INFO:teuthology.orchestra.run.smithi149.stdout:Description : 2024-09-10T18:43:57.593 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T18:43:57.593 INFO:teuthology.orchestra.run.smithi149.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-10T18:43:57.593 INFO:teuthology.orchestra.run.smithi149.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-10T18:43:57.593 INFO:teuthology.orchestra.run.smithi149.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-10T18:43:57.593 INFO:teuthology.orchestra.run.smithi149.stdout:hardware devices. 2024-09-10T18:43:57.593 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T18:43:57.594 INFO:teuthology.orchestra.run.smithi149.stdout:This subpackage provides tools for support of all platforms. 2024-09-10T18:43:57.595 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-10T18:43:57.595 INFO:teuthology.task.kernel:Updating grub on smithi149 to boot 5.14.0-505.el9.x86_64 2024-09-10T18:43:57.595 DEBUG:teuthology.orchestra.run.smithi149:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-10T18:43:58.280 INFO:teuthology.orchestra.run.smithi149.stderr:Generating grub configuration file ... 2024-09-10T18:43:58.475 INFO:teuthology.orchestra.run.smithi047.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-10T18:43:58.503 INFO:teuthology.orchestra.run.smithi047.stderr:done 2024-09-10T18:43:58.505 DEBUG:teuthology.orchestra.run.smithi047:> mktemp 2024-09-10T18:43:58.520 INFO:teuthology.orchestra.run.smithi047.stdout:/tmp/tmp.I4OxJZNsaZ 2024-09-10T18:43:58.521 DEBUG:teuthology.orchestra.run.smithi047:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.I4OxJZNsaZ 2024-09-10T18:43:58.587 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 0666 /tmp/tmp.I4OxJZNsaZ 2024-09-10T18:43:58.729 DEBUG:teuthology.orchestra.remote:smithi047:/tmp/tmp.I4OxJZNsaZ is 6KB 2024-09-10T18:43:58.778 DEBUG:teuthology.orchestra.run.smithi047:> rm -fr /tmp/tmp.I4OxJZNsaZ 2024-09-10T18:43:58.793 DEBUG:teuthology.orchestra.run.smithi047:> sudo /bin/ls /boot/loader/entries || true 2024-09-10T18:43:58.870 INFO:teuthology.orchestra.run.smithi047.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-10T18:43:58.870 INFO:teuthology.orchestra.run.smithi047.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-10T18:43:58.870 INFO:teuthology.orchestra.run.smithi047.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-10T18:43:58.870 INFO:teuthology.orchestra.run.smithi047.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-10T18:43:58.871 DEBUG:teuthology.orchestra.run.smithi047:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-10T18:43:59.034 DEBUG:teuthology.orchestra.run.smithi047:> sudo shutdown -r now 2024-09-10T18:43:59.637 INFO:teuthology.orchestra.run.smithi149.stderr:File descriptor 3 (pipe:[42791]) leaked on vgs invocation. Parent PID 22675: grub2-probe 2024-09-10T18:43:59.638 INFO:teuthology.orchestra.run.smithi149.stderr:File descriptor 9 (pipe:[43628]) leaked on vgs invocation. Parent PID 22675: grub2-probe 2024-09-10T18:43:59.685 INFO:teuthology.orchestra.run.smithi149.stderr:File descriptor 3 (pipe:[42791]) leaked on vgs invocation. Parent PID 22675: grub2-probe 2024-09-10T18:43:59.685 INFO:teuthology.orchestra.run.smithi149.stderr:File descriptor 9 (pipe:[43628]) leaked on vgs invocation. Parent PID 22675: grub2-probe 2024-09-10T18:43:59.836 INFO:teuthology.orchestra.run.smithi149.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-10T18:43:59.859 INFO:teuthology.orchestra.run.smithi149.stderr:done 2024-09-10T18:43:59.861 DEBUG:teuthology.orchestra.run.smithi149:> mktemp 2024-09-10T18:43:59.877 INFO:teuthology.orchestra.run.smithi149.stdout:/tmp/tmp.9HmjERlbOd 2024-09-10T18:43:59.877 DEBUG:teuthology.orchestra.run.smithi149:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.9HmjERlbOd 2024-09-10T18:43:59.944 DEBUG:teuthology.orchestra.run.smithi149:> sudo chmod 0666 /tmp/tmp.9HmjERlbOd 2024-09-10T18:44:00.096 DEBUG:teuthology.orchestra.remote:smithi149:/tmp/tmp.9HmjERlbOd is 6KB 2024-09-10T18:44:00.146 DEBUG:teuthology.orchestra.run.smithi149:> rm -fr /tmp/tmp.9HmjERlbOd 2024-09-10T18:44:00.160 DEBUG:teuthology.orchestra.run.smithi149:> sudo /bin/ls /boot/loader/entries || true 2024-09-10T18:44:00.228 INFO:teuthology.orchestra.run.smithi149.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-10T18:44:00.228 INFO:teuthology.orchestra.run.smithi149.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-10T18:44:00.228 INFO:teuthology.orchestra.run.smithi149.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-10T18:44:00.229 INFO:teuthology.orchestra.run.smithi149.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-10T18:44:00.230 DEBUG:teuthology.orchestra.run.smithi149:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-10T18:44:00.371 DEBUG:teuthology.orchestra.run.smithi149:> sudo shutdown -r now 2024-09-10T18:44:29.065 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-10T18:44:29.066 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi047.front.sepia.ceph.com' 2024-09-10T18:44:29.068 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T18:44:30.375 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-10T18:44:30.375 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi149.front.sepia.ceph.com' 2024-09-10T18:44:30.376 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T18:44:47.549 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.47 2024-09-10T18:44:48.829 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.149 2024-09-10T18:44:56.557 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi047.front.sepia.ceph.com' 2024-09-10T18:44:56.559 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T18:44:57.832 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi149.front.sepia.ceph.com' 2024-09-10T18:44:57.833 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T18:45:00.893 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.149 2024-09-10T18:45:12.903 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi149.front.sepia.ceph.com' 2024-09-10T18:45:12.904 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T18:45:56.601 DEBUG:teuthology.orchestra.remote:timed out 2024-09-10T18:46:08.613 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi047.front.sepia.ceph.com' 2024-09-10T18:46:08.615 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T18:46:09.034 DEBUG:teuthology.orchestra.run.smithi047:> true 2024-09-10T18:46:09.379 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi047.front.sepia.ceph.com' 2024-09-10T18:46:09.379 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-505.el9.x86_64"... 2024-09-10T18:46:09.380 DEBUG:teuthology.orchestra.run.smithi047:> uname -r 2024-09-10T18:46:09.397 INFO:teuthology.orchestra.run.smithi047.stdout:5.14.0-505.el9.x86_64 2024-09-10T18:46:09.397 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-10T18:46:09.397 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-10T18:46:09.398 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-10T18:46:10.398 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-10T18:46:10.399 DEBUG:teuthology.orchestra.run.smithi047:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-10T18:46:10.472 INFO:teuthology.orchestra.run.smithi047.stdout:ttyS1 2024-09-10T18:46:10.498 DEBUG:teuthology.parallel:result is None 2024-09-10T18:46:12.906 DEBUG:teuthology.orchestra.remote:timed out 2024-09-10T18:46:27.907 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi149.front.sepia.ceph.com' 2024-09-10T18:46:27.908 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T18:46:28.308 DEBUG:teuthology.orchestra.run.smithi149:> true 2024-09-10T18:46:28.658 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi149.front.sepia.ceph.com' 2024-09-10T18:46:28.659 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-505.el9.x86_64"... 2024-09-10T18:46:28.659 DEBUG:teuthology.orchestra.run.smithi149:> uname -r 2024-09-10T18:46:28.675 INFO:teuthology.orchestra.run.smithi149.stdout:5.14.0-505.el9.x86_64 2024-09-10T18:46:28.675 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-10T18:46:28.675 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-10T18:46:28.675 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-10T18:46:29.676 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-10T18:46:29.676 DEBUG:teuthology.orchestra.run.smithi149:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-10T18:46:29.753 INFO:teuthology.orchestra.run.smithi149.stdout:ttyS1 2024-09-10T18:46:29.774 DEBUG:teuthology.parallel:result is None 2024-09-10T18:46:29.774 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-10T18:46:29.782 INFO:teuthology.task.internal:Creating test directory... 2024-09-10T18:46:29.782 DEBUG:teuthology.orchestra.run.smithi047:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-10T18:46:29.785 DEBUG:teuthology.orchestra.run.smithi149:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-10T18:46:29.803 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-10T18:46:29.809 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-10T18:46:29.817 INFO:teuthology.task.internal:Creating archive directory... 2024-09-10T18:46:29.818 DEBUG:teuthology.orchestra.run.smithi047:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-10T18:46:29.844 DEBUG:teuthology.orchestra.run.smithi149:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-10T18:46:29.876 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-10T18:46:29.882 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-10T18:46:29.883 DEBUG:teuthology.orchestra.run.smithi047:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-10T18:46:29.907 DEBUG:teuthology.orchestra.run.smithi149:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-10T18:46:29.941 INFO:teuthology.orchestra.run.smithi047.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T18:46:29.947 INFO:teuthology.orchestra.run.smithi149.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T18:46:29.953 INFO:teuthology.orchestra.run.smithi047.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T18:46:29.958 INFO:teuthology.orchestra.run.smithi149.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T18:46:29.960 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-10T18:46:29.966 INFO:teuthology.task.internal:Configuring sudo... 2024-09-10T18:46:29.967 DEBUG:teuthology.orchestra.run.smithi047:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-10T18:46:30.000 DEBUG:teuthology.orchestra.run.smithi149:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-10T18:46:30.028 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-10T18:46:30.036 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-10T18:46:30.037 DEBUG:teuthology.orchestra.run.smithi047:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-10T18:46:30.070 DEBUG:teuthology.orchestra.run.smithi149:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-10T18:46:30.087 DEBUG:teuthology.orchestra.run.smithi047:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T18:46:30.170 DEBUG:teuthology.orchestra.run.smithi047:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T18:46:30.221 DEBUG:teuthology.orchestra.run.smithi047:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T18:46:30.307 DEBUG:teuthology.orchestra.run.smithi047:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T18:46:30.352 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-09-10T18:46:30.353 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-10T18:46:30.422 DEBUG:teuthology.orchestra.run.smithi149:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T18:46:30.470 DEBUG:teuthology.orchestra.run.smithi149:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T18:46:30.529 DEBUG:teuthology.orchestra.run.smithi149:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T18:46:30.573 DEBUG:teuthology.orchestra.run.smithi149:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T18:46:30.616 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-09-10T18:46:30.616 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-10T18:46:30.683 DEBUG:teuthology.orchestra.run.smithi047:> sudo service rsyslog restart 2024-09-10T18:46:30.686 DEBUG:teuthology.orchestra.run.smithi149:> sudo service rsyslog restart 2024-09-10T18:46:30.742 INFO:teuthology.orchestra.run.smithi047.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T18:46:30.771 INFO:teuthology.orchestra.run.smithi149.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T18:46:31.077 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-10T18:46:31.084 INFO:teuthology.task.internal:Starting timer... 2024-09-10T18:46:31.084 INFO:teuthology.run_tasks:Running task pcp... 2024-09-10T18:46:31.107 INFO:teuthology.run_tasks:Running task selinux... 2024-09-10T18:46:31.115 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-09-10T18:46:31.115 DEBUG:teuthology.orchestra.run.smithi047:> sudo service auditd rotate 2024-09-10T18:46:31.164 INFO:teuthology.orchestra.run.smithi047.stdout:Rotating logs: 2024-09-10T18:46:31.165 DEBUG:teuthology.orchestra.run.smithi149:> sudo service auditd rotate 2024-09-10T18:46:31.224 INFO:teuthology.orchestra.run.smithi149.stdout:Rotating logs: 2024-09-10T18:46:31.226 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-10T18:46:31.226 DEBUG:teuthology.orchestra.run.smithi047:> /usr/sbin/getenforce 2024-09-10T18:46:31.254 INFO:teuthology.orchestra.run.smithi047.stdout:Permissive 2024-09-10T18:46:31.254 DEBUG:teuthology.orchestra.run.smithi149:> /usr/sbin/getenforce 2024-09-10T18:46:31.274 INFO:teuthology.orchestra.run.smithi149.stdout:Permissive 2024-09-10T18:46:31.274 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi047.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi149.front.sepia.ceph.com': 'permissive'} 2024-09-10T18:46:31.274 DEBUG:teuthology.orchestra.run.smithi047:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-10T18:46:31.322 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:46:31.323 DEBUG:teuthology.orchestra.run.smithi149:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-10T18:46:31.350 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:46:31.351 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-10T18:46:31.351 DEBUG:teuthology.orchestra.run.smithi047:> sudo /usr/sbin/setenforce permissive 2024-09-10T18:46:31.396 DEBUG:teuthology.orchestra.run.smithi149:> sudo /usr/sbin/setenforce permissive 2024-09-10T18:46:31.433 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-10T18:46:31.443 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-10T18:46:31.446 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-10T18:46:31.568 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-10T18:46:31.584 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-10T18:46:31.585 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi047.front.sepia.ceph.com,smithi149.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-10T18:50:21.119 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi047.front.sepia.ceph.com'), Remote(name='ubuntu@smithi149.front.sepia.ceph.com')] 2024-09-10T18:50:21.120 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi047.front.sepia.ceph.com' 2024-09-10T18:50:21.126 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T18:50:21.205 DEBUG:teuthology.orchestra.run.smithi047:> true 2024-09-10T18:50:21.294 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi047.front.sepia.ceph.com' 2024-09-10T18:50:21.294 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi149.front.sepia.ceph.com' 2024-09-10T18:50:21.295 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T18:50:21.371 DEBUG:teuthology.orchestra.run.smithi149:> true 2024-09-10T18:50:21.448 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi149.front.sepia.ceph.com' 2024-09-10T18:50:21.449 INFO:teuthology.run_tasks:Running task clock... 2024-09-10T18:50:21.459 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-10T18:50:21.460 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-10T18:50:21.460 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T18:50:21.463 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-10T18:50:21.463 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T18:50:21.494 INFO:teuthology.orchestra.run.smithi047.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-10T18:50:21.510 INFO:teuthology.orchestra.run.smithi047.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-10T18:50:21.518 INFO:teuthology.orchestra.run.smithi149.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-10T18:50:21.533 INFO:teuthology.orchestra.run.smithi149.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-10T18:50:21.537 INFO:teuthology.orchestra.run.smithi047.stderr:sudo: ntpd: command not found 2024-09-10T18:50:21.549 INFO:teuthology.orchestra.run.smithi047.stdout:506 Cannot talk to daemon 2024-09-10T18:50:21.557 INFO:teuthology.orchestra.run.smithi149.stderr:sudo: ntpd: command not found 2024-09-10T18:50:21.564 INFO:teuthology.orchestra.run.smithi047.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-10T18:50:21.568 INFO:teuthology.orchestra.run.smithi149.stdout:506 Cannot talk to daemon 2024-09-10T18:50:21.580 INFO:teuthology.orchestra.run.smithi047.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-10T18:50:21.583 INFO:teuthology.orchestra.run.smithi149.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-10T18:50:21.597 INFO:teuthology.orchestra.run.smithi149.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-10T18:50:21.639 INFO:teuthology.orchestra.run.smithi047.stderr:bash: line 1: ntpq: command not found 2024-09-10T18:50:21.643 INFO:teuthology.orchestra.run.smithi047.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T18:50:21.643 INFO:teuthology.orchestra.run.smithi047.stdout:=============================================================================== 2024-09-10T18:50:21.643 INFO:teuthology.orchestra.run.smithi047.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T18:50:21.643 INFO:teuthology.orchestra.run.smithi047.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T18:50:21.643 INFO:teuthology.orchestra.run.smithi047.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T18:50:21.643 INFO:teuthology.orchestra.run.smithi047.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T18:50:21.651 INFO:teuthology.orchestra.run.smithi149.stderr:bash: line 1: ntpq: command not found 2024-09-10T18:50:21.655 INFO:teuthology.orchestra.run.smithi149.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T18:50:21.656 INFO:teuthology.orchestra.run.smithi149.stdout:=============================================================================== 2024-09-10T18:50:21.656 INFO:teuthology.orchestra.run.smithi149.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T18:50:21.656 INFO:teuthology.orchestra.run.smithi149.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T18:50:21.656 INFO:teuthology.orchestra.run.smithi149.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T18:50:21.656 INFO:teuthology.orchestra.run.smithi149.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T18:50:21.656 INFO:teuthology.run_tasks:Running task pexec... 2024-09-10T18:50:21.665 INFO:teuthology.task.pexec:Executing custom commands... 2024-09-10T18:50:21.666 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi047.front.sepia.ceph.com 2024-09-10T18:50:21.666 DEBUG:teuthology.orchestra.run.smithi047:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-09-10T18:50:21.666 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi149.front.sepia.ceph.com 2024-09-10T18:50:21.666 DEBUG:teuthology.orchestra.run.smithi149:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-09-10T18:50:22.051 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-09-10T18:50:22.051 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-09-10T18:50:22.057 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-09-10T18:50:22.057 INFO:teuthology.orchestra.run.smithi149.stdout: Package Architecture Version Repository Size 2024-09-10T18:50:22.057 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-09-10T18:50:22.057 INFO:teuthology.orchestra.run.smithi149.stdout:Removing: 2024-09-10T18:50:22.057 INFO:teuthology.orchestra.run.smithi149.stdout: nvme-cli x86_64 2.9.1-6.el9 @baseos 5.3 M 2024-09-10T18:50:22.057 INFO:teuthology.orchestra.run.smithi149.stdout:Removing unused dependencies: 2024-09-10T18:50:22.057 INFO:teuthology.orchestra.run.smithi149.stdout: libnvme x86_64 1.9-3.el9 @baseos 273 k 2024-09-10T18:50:22.057 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T18:50:22.058 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction Summary 2024-09-10T18:50:22.058 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-09-10T18:50:22.058 INFO:teuthology.orchestra.run.smithi149.stdout:Remove 2 Packages 2024-09-10T18:50:22.058 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T18:50:22.058 INFO:teuthology.orchestra.run.smithi149.stdout:Freed space: 5.5 M 2024-09-10T18:50:22.058 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction check 2024-09-10T18:50:22.062 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction check succeeded. 2024-09-10T18:50:22.062 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction test 2024-09-10T18:50:22.063 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-10T18:50:22.063 INFO:teuthology.orchestra.run.smithi047.stdout: Package Architecture Version Repository Size 2024-09-10T18:50:22.063 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-10T18:50:22.063 INFO:teuthology.orchestra.run.smithi047.stdout:Removing: 2024-09-10T18:50:22.063 INFO:teuthology.orchestra.run.smithi047.stdout: nvme-cli x86_64 2.9.1-6.el9 @baseos 5.3 M 2024-09-10T18:50:22.063 INFO:teuthology.orchestra.run.smithi047.stdout:Removing unused dependencies: 2024-09-10T18:50:22.063 INFO:teuthology.orchestra.run.smithi047.stdout: libnvme x86_64 1.9-3.el9 @baseos 273 k 2024-09-10T18:50:22.063 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:50:22.063 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction Summary 2024-09-10T18:50:22.064 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-10T18:50:22.064 INFO:teuthology.orchestra.run.smithi047.stdout:Remove 2 Packages 2024-09-10T18:50:22.064 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:50:22.064 INFO:teuthology.orchestra.run.smithi047.stdout:Freed space: 5.5 M 2024-09-10T18:50:22.064 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction check 2024-09-10T18:50:22.068 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction check succeeded. 2024-09-10T18:50:22.068 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction test 2024-09-10T18:50:22.129 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction test succeeded. 2024-09-10T18:50:22.130 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction 2024-09-10T18:50:22.133 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction test succeeded. 2024-09-10T18:50:22.133 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction 2024-09-10T18:50:22.282 INFO:teuthology.orchestra.run.smithi047.stdout: Preparing : 1/1 2024-09-10T18:50:22.300 INFO:teuthology.orchestra.run.smithi149.stdout: Preparing : 1/1 2024-09-10T18:50:22.366 INFO:teuthology.orchestra.run.smithi149.stdout: Erasing : nvme-cli-2.9.1-6.el9.x86_64 1/2 2024-09-10T18:50:22.379 INFO:teuthology.orchestra.run.smithi047.stdout: Erasing : nvme-cli-2.9.1-6.el9.x86_64 1/2 2024-09-10T18:50:22.423 INFO:teuthology.orchestra.run.smithi149.stdout: Erasing : libnvme-1.9-3.el9.x86_64 2/2 2024-09-10T18:50:22.440 INFO:teuthology.orchestra.run.smithi047.stdout: Erasing : libnvme-1.9-3.el9.x86_64 2/2 2024-09-10T18:50:22.867 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: libnvme-1.9-3.el9.x86_64 2/2 2024-09-10T18:50:22.867 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/2 2024-09-10T18:50:22.909 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: libnvme-1.9-3.el9.x86_64 2/2 2024-09-10T18:50:22.909 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/2 2024-09-10T18:50:23.122 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/2 2024-09-10T18:50:23.122 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:50:23.122 INFO:teuthology.orchestra.run.smithi047.stdout:Removed: 2024-09-10T18:50:23.122 INFO:teuthology.orchestra.run.smithi047.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-10T18:50:23.122 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:50:23.122 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-09-10T18:50:23.171 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/2 2024-09-10T18:50:23.171 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T18:50:23.171 INFO:teuthology.orchestra.run.smithi149.stdout:Removed: 2024-09-10T18:50:23.171 INFO:teuthology.orchestra.run.smithi149.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-10T18:50:23.171 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T18:50:23.171 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-09-10T18:50:23.951 INFO:teuthology.orchestra.run.smithi047.stdout:Last metadata expiration check: 0:00:49 ago on Tue 10 Sep 2024 06:49:34 PM UTC. 2024-09-10T18:50:23.958 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:00:53 ago on Tue 10 Sep 2024 06:49:30 PM UTC. 2024-09-10T18:50:24.087 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-09-10T18:50:24.088 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-10T18:50:24.088 INFO:teuthology.orchestra.run.smithi047.stdout: Package Architecture Version Repository Size 2024-09-10T18:50:24.088 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-10T18:50:24.088 INFO:teuthology.orchestra.run.smithi047.stdout:Installing: 2024-09-10T18:50:24.088 INFO:teuthology.orchestra.run.smithi047.stdout: nvme-cli x86_64 2.9.1-6.el9 baseos 906 k 2024-09-10T18:50:24.088 INFO:teuthology.orchestra.run.smithi047.stdout: nvmetcli noarch 0.7-3.el9 baseos 44 k 2024-09-10T18:50:24.088 INFO:teuthology.orchestra.run.smithi047.stdout:Installing dependencies: 2024-09-10T18:50:24.088 INFO:teuthology.orchestra.run.smithi047.stdout: libnvme x86_64 1.9-3.el9 baseos 106 k 2024-09-10T18:50:24.088 INFO:teuthology.orchestra.run.smithi047.stdout: python3-configshell noarch 1:1.1.30-1.el9 baseos 72 k 2024-09-10T18:50:24.088 INFO:teuthology.orchestra.run.smithi047.stdout: python3-kmod x86_64 0.9-32.el9 baseos 84 k 2024-09-10T18:50:24.088 INFO:teuthology.orchestra.run.smithi047.stdout: python3-urwid x86_64 2.1.2-4.el9 baseos 837 k 2024-09-10T18:50:24.089 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:50:24.089 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction Summary 2024-09-10T18:50:24.089 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-10T18:50:24.089 INFO:teuthology.orchestra.run.smithi047.stdout:Install 6 Packages 2024-09-10T18:50:24.089 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:50:24.094 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-09-10T18:50:24.095 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-09-10T18:50:24.095 INFO:teuthology.orchestra.run.smithi149.stdout: Package Architecture Version Repository Size 2024-09-10T18:50:24.095 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-09-10T18:50:24.095 INFO:teuthology.orchestra.run.smithi149.stdout:Installing: 2024-09-10T18:50:24.095 INFO:teuthology.orchestra.run.smithi149.stdout: nvme-cli x86_64 2.9.1-6.el9 baseos 906 k 2024-09-10T18:50:24.095 INFO:teuthology.orchestra.run.smithi149.stdout: nvmetcli noarch 0.7-3.el9 baseos 44 k 2024-09-10T18:50:24.095 INFO:teuthology.orchestra.run.smithi149.stdout:Installing dependencies: 2024-09-10T18:50:24.095 INFO:teuthology.orchestra.run.smithi149.stdout: libnvme x86_64 1.9-3.el9 baseos 106 k 2024-09-10T18:50:24.095 INFO:teuthology.orchestra.run.smithi149.stdout: python3-configshell noarch 1:1.1.30-1.el9 baseos 72 k 2024-09-10T18:50:24.096 INFO:teuthology.orchestra.run.smithi149.stdout: python3-kmod x86_64 0.9-32.el9 baseos 84 k 2024-09-10T18:50:24.096 INFO:teuthology.orchestra.run.smithi149.stdout: python3-urwid x86_64 2.1.2-4.el9 baseos 837 k 2024-09-10T18:50:24.096 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T18:50:24.096 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction Summary 2024-09-10T18:50:24.096 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-09-10T18:50:24.096 INFO:teuthology.orchestra.run.smithi149.stdout:Install 6 Packages 2024-09-10T18:50:24.096 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T18:50:24.100 INFO:teuthology.orchestra.run.smithi149.stdout:Total download size: 2.0 M 2024-09-10T18:50:24.100 INFO:teuthology.orchestra.run.smithi149.stdout:Installed size: 9.2 M 2024-09-10T18:50:24.100 INFO:teuthology.orchestra.run.smithi149.stdout:Downloading Packages: 2024-09-10T18:50:24.104 INFO:teuthology.orchestra.run.smithi047.stdout:Total download size: 2.0 M 2024-09-10T18:50:24.104 INFO:teuthology.orchestra.run.smithi047.stdout:Installed size: 9.2 M 2024-09-10T18:50:24.104 INFO:teuthology.orchestra.run.smithi047.stdout:Downloading Packages: 2024-09-10T18:50:24.238 INFO:teuthology.orchestra.run.smithi149.stdout:(1/6): nvmetcli-0.7-3.el9.noarch.rpm 397 kB/s | 44 kB 00:00 2024-09-10T18:50:24.243 INFO:teuthology.orchestra.run.smithi047.stdout:(1/6): nvmetcli-0.7-3.el9.noarch.rpm 490 kB/s | 44 kB 00:00 2024-09-10T18:50:24.263 INFO:teuthology.orchestra.run.smithi149.stdout:(2/6): libnvme-1.9-3.el9.x86_64.rpm 772 kB/s | 106 kB 00:00 2024-09-10T18:50:24.269 INFO:teuthology.orchestra.run.smithi047.stdout:(2/6): libnvme-1.9-3.el9.x86_64.rpm 913 kB/s | 106 kB 00:00 2024-09-10T18:50:24.294 INFO:teuthology.orchestra.run.smithi047.stdout:(3/6): python3-configshell-1.1.30-1.el9.noarch. 1.4 MB/s | 72 kB 00:00 2024-09-10T18:50:24.313 INFO:teuthology.orchestra.run.smithi149.stdout:(3/6): python3-kmod-0.9-32.el9.x86_64.rpm 1.7 MB/s | 84 kB 00:00 2024-09-10T18:50:24.319 INFO:teuthology.orchestra.run.smithi047.stdout:(4/6): python3-kmod-0.9-32.el9.x86_64.rpm 1.6 MB/s | 84 kB 00:00 2024-09-10T18:50:24.330 INFO:teuthology.orchestra.run.smithi149.stdout:(4/6): python3-configshell-1.1.30-1.el9.noarch. 779 kB/s | 72 kB 00:00 2024-09-10T18:50:24.353 INFO:teuthology.orchestra.run.smithi047.stdout:(5/6): nvme-cli-2.9.1-6.el9.x86_64.rpm 4.4 MB/s | 906 kB 00:00 2024-09-10T18:50:24.420 INFO:teuthology.orchestra.run.smithi047.stdout:(6/6): python3-urwid-2.1.2-4.el9.x86_64.rpm 6.5 MB/s | 837 kB 00:00 2024-09-10T18:50:24.420 INFO:teuthology.orchestra.run.smithi047.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:50:24.420 INFO:teuthology.orchestra.run.smithi047.stdout:Total 6.3 MB/s | 2.0 MB 00:00 2024-09-10T18:50:24.480 INFO:teuthology.orchestra.run.smithi149.stdout:(5/6): nvme-cli-2.9.1-6.el9.x86_64.rpm 2.5 MB/s | 906 kB 00:00 2024-09-10T18:50:24.495 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction check 2024-09-10T18:50:24.510 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction check succeeded. 2024-09-10T18:50:24.510 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction test 2024-09-10T18:50:24.705 INFO:teuthology.orchestra.run.smithi149.stdout:(6/6): python3-urwid-2.1.2-4.el9.x86_64.rpm 2.1 MB/s | 837 kB 00:00 2024-09-10T18:50:24.706 INFO:teuthology.orchestra.run.smithi149.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:50:24.706 INFO:teuthology.orchestra.run.smithi149.stdout:Total 3.3 MB/s | 2.0 MB 00:00 2024-09-10T18:50:24.779 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction check 2024-09-10T18:50:24.793 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction check succeeded. 2024-09-10T18:50:24.793 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction test 2024-09-10T18:50:24.814 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction test succeeded. 2024-09-10T18:50:24.814 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction 2024-09-10T18:50:25.076 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction test succeeded. 2024-09-10T18:50:25.077 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction 2024-09-10T18:50:25.258 INFO:teuthology.orchestra.run.smithi047.stdout: Preparing : 1/1 2024-09-10T18:50:25.357 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-urwid-2.1.2-4.el9.x86_64 1/6 2024-09-10T18:50:25.391 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-configshell-1:1.1.30-1.el9.noarch 2/6 2024-09-10T18:50:25.420 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-kmod-0.9-32.el9.x86_64 3/6 2024-09-10T18:50:25.487 INFO:teuthology.orchestra.run.smithi149.stdout: Preparing : 1/1 2024-09-10T18:50:25.577 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-urwid-2.1.2-4.el9.x86_64 1/6 2024-09-10T18:50:25.611 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-configshell-1:1.1.30-1.el9.noarch 2/6 2024-09-10T18:50:25.640 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-kmod-0.9-32.el9.x86_64 3/6 2024-09-10T18:50:25.649 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libnvme-1.9-3.el9.x86_64 4/6 2024-09-10T18:50:25.692 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : nvme-cli-2.9.1-6.el9.x86_64 5/6 2024-09-10T18:50:25.845 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libnvme-1.9-3.el9.x86_64 4/6 2024-09-10T18:50:25.880 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : nvme-cli-2.9.1-6.el9.x86_64 5/6 2024-09-10T18:50:26.169 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: nvme-cli-2.9.1-6.el9.x86_64 5/6 2024-09-10T18:50:26.195 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : nvmetcli-0.7-3.el9.noarch 6/6 2024-09-10T18:50:26.314 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: nvme-cli-2.9.1-6.el9.x86_64 5/6 2024-09-10T18:50:26.340 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : nvmetcli-0.7-3.el9.noarch 6/6 2024-09-10T18:50:27.148 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: nvmetcli-0.7-3.el9.noarch 6/6 2024-09-10T18:50:27.148 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/6 2024-09-10T18:50:27.148 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/6 2024-09-10T18:50:27.148 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : nvmetcli-0.7-3.el9.noarch 3/6 2024-09-10T18:50:27.148 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-configshell-1:1.1.30-1.el9.noarch 4/6 2024-09-10T18:50:27.148 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-kmod-0.9-32.el9.x86_64 5/6 2024-09-10T18:50:27.322 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: nvmetcli-0.7-3.el9.noarch 6/6 2024-09-10T18:50:27.323 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/6 2024-09-10T18:50:27.323 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/6 2024-09-10T18:50:27.323 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : nvmetcli-0.7-3.el9.noarch 3/6 2024-09-10T18:50:27.323 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-configshell-1:1.1.30-1.el9.noarch 4/6 2024-09-10T18:50:27.323 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-kmod-0.9-32.el9.x86_64 5/6 2024-09-10T18:50:27.521 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-urwid-2.1.2-4.el9.x86_64 6/6 2024-09-10T18:50:27.521 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:50:27.522 INFO:teuthology.orchestra.run.smithi047.stdout:Installed: 2024-09-10T18:50:27.522 INFO:teuthology.orchestra.run.smithi047.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-10T18:50:27.522 INFO:teuthology.orchestra.run.smithi047.stdout: nvmetcli-0.7-3.el9.noarch python3-configshell-1:1.1.30-1.el9.noarch 2024-09-10T18:50:27.522 INFO:teuthology.orchestra.run.smithi047.stdout: python3-kmod-0.9-32.el9.x86_64 python3-urwid-2.1.2-4.el9.x86_64 2024-09-10T18:50:27.522 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:50:27.522 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-09-10T18:50:27.628 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-urwid-2.1.2-4.el9.x86_64 6/6 2024-09-10T18:50:27.628 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T18:50:27.628 INFO:teuthology.orchestra.run.smithi149.stdout:Installed: 2024-09-10T18:50:27.629 INFO:teuthology.orchestra.run.smithi149.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-10T18:50:27.629 INFO:teuthology.orchestra.run.smithi149.stdout: nvmetcli-0.7-3.el9.noarch python3-configshell-1:1.1.30-1.el9.noarch 2024-09-10T18:50:27.629 INFO:teuthology.orchestra.run.smithi149.stdout: python3-kmod-0.9-32.el9.x86_64 python3-urwid-2.1.2-4.el9.x86_64 2024-09-10T18:50:27.629 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T18:50:27.629 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-09-10T18:50:27.843 DEBUG:teuthology.parallel:result is None 2024-09-10T18:50:27.926 DEBUG:teuthology.parallel:result is None 2024-09-10T18:50:27.927 INFO:teuthology.run_tasks:Running task install... 2024-09-10T18:50:27.935 DEBUG:teuthology.task.install:project ceph 2024-09-10T18:50:27.936 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '5be57395ea235535910512ae73ba8e7ff25fe588'}} 2024-09-10T18:50:27.936 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '5be57395ea235535910512ae73ba8e7ff25fe588'} 2024-09-10T18:50:27.936 INFO:teuthology.task.install:Using flavor: default 2024-09-10T18:50:27.941 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-10T18:50:27.941 INFO:teuthology.task.install:extra packages: [] 2024-09-10T18:50:27.942 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '5be57395ea235535910512ae73ba8e7ff25fe588', 'tag': None, 'wait_for_package': False} 2024-09-10T18:50:27.942 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5be57395ea235535910512ae73ba8e7ff25fe588 2024-09-10T18:50:27.944 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '5be57395ea235535910512ae73ba8e7ff25fe588', 'tag': None, 'wait_for_package': False} 2024-09-10T18:50:27.944 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5be57395ea235535910512ae73ba8e7ff25fe588 2024-09-10T18:50:28.087 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/centos/9/flavors/default/ 2024-09-10T18:50:28.087 INFO:teuthology.task.install.rpm:Package version is 17.2.7-1606.g5be57395 2024-09-10T18:50:28.093 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/centos/9/flavors/default/ 2024-09-10T18:50:28.094 INFO:teuthology.task.install.rpm:Package version is 17.2.7-1606.g5be57395 2024-09-10T18:50:28.206 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-10T18:50:28.206 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-09-10T18:50:28.206 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-10T18:50:28.236 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-10T18:50:28.236 DEBUG:teuthology.orchestra.run.smithi149:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/5be57395ea235535910512ae73ba8e7ff25fe588/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-10T18:50:28.239 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-10T18:50:28.239 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-09-10T18:50:28.239 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-10T18:50:28.270 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-10T18:50:28.270 DEBUG:teuthology.orchestra.run.smithi047:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/5be57395ea235535910512ae73ba8e7ff25fe588/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-10T18:50:28.302 DEBUG:teuthology.orchestra.run.smithi149:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-10T18:50:28.339 DEBUG:teuthology.orchestra.run.smithi047:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-10T18:50:28.379 DEBUG:teuthology.orchestra.run.smithi149:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-10T18:50:28.417 DEBUG:teuthology.orchestra.run.smithi047:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-10T18:50:28.444 INFO:teuthology.orchestra.run.smithi149.stdout:check_obsoletes = 1 2024-09-10T18:50:28.446 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum clean all 2024-09-10T18:50:28.483 INFO:teuthology.orchestra.run.smithi047.stdout:check_obsoletes = 1 2024-09-10T18:50:28.485 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum clean all 2024-09-10T18:50:28.670 INFO:teuthology.orchestra.run.smithi149.stdout:45 files removed 2024-09-10T18:50:28.698 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install ceph-radosgw 2024-09-10T18:50:28.723 INFO:teuthology.orchestra.run.smithi047.stdout:45 files removed 2024-09-10T18:50:28.750 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install ceph-radosgw 2024-09-10T18:50:29.313 INFO:teuthology.orchestra.run.smithi149.stdout:ceph packages for x86_64 240 kB/s | 77 kB 00:00 2024-09-10T18:50:29.343 INFO:teuthology.orchestra.run.smithi047.stdout:ceph packages for x86_64 265 kB/s | 77 kB 00:00 2024-09-10T18:50:29.598 INFO:teuthology.orchestra.run.smithi149.stdout:ceph noarch packages 45 kB/s | 11 kB 00:00 2024-09-10T18:50:29.618 INFO:teuthology.orchestra.run.smithi047.stdout:ceph noarch packages 47 kB/s | 11 kB 00:00 2024-09-10T18:50:29.873 INFO:teuthology.orchestra.run.smithi149.stdout:ceph source packages 7.6 kB/s | 1.8 kB 00:00 2024-09-10T18:50:29.877 INFO:teuthology.orchestra.run.smithi047.stdout:ceph source packages 8.1 kB/s | 1.8 kB 00:00 2024-09-10T18:50:30.381 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS Stream 9 - BaseOS 17 MB/s | 8.2 MB 00:00 2024-09-10T18:50:33.286 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - BaseOS 2.4 MB/s | 8.2 MB 00:03 2024-09-10T18:50:34.674 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS Stream 9 - AppStream 6.4 MB/s | 20 MB 00:03 2024-09-10T18:50:35.212 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - AppStream 26 MB/s | 20 MB 00:00 2024-09-10T18:50:39.588 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - CRB 15 MB/s | 6.5 MB 00:00 2024-09-10T18:50:40.967 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS Stream 9 - CRB 2.6 MB/s | 6.5 MB 00:02 2024-09-10T18:50:41.297 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - Extras packages 44 kB/s | 19 kB 00:00 2024-09-10T18:50:42.093 INFO:teuthology.orchestra.run.smithi047.stdout:Extra Packages for Enterprise Linux 34 MB/s | 23 MB 00:00 2024-09-10T18:50:42.359 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS Stream 9 - Extras packages 105 kB/s | 19 kB 00:00 2024-09-10T18:50:43.247 INFO:teuthology.orchestra.run.smithi149.stdout:Extra Packages for Enterprise Linux 30 MB/s | 23 MB 00:00 2024-09-10T18:50:47.858 INFO:teuthology.orchestra.run.smithi047.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-10T18:50:48.761 INFO:teuthology.orchestra.run.smithi149.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-10T18:50:50.342 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-09-10T18:50:50.344 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-10T18:50:50.344 INFO:teuthology.orchestra.run.smithi047.stdout: Package Arch Version Repository Size 2024-09-10T18:50:50.344 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-10T18:50:50.344 INFO:teuthology.orchestra.run.smithi047.stdout:Installing: 2024-09-10T18:50:50.344 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-radosgw x86_64 2:17.2.7-1606.g5be57395.el9 ceph 14 M 2024-09-10T18:50:50.344 INFO:teuthology.orchestra.run.smithi047.stdout:Upgrading: 2024-09-10T18:50:50.344 INFO:teuthology.orchestra.run.smithi047.stdout: librados2 x86_64 2:17.2.7-1606.g5be57395.el9 ceph 3.1 M 2024-09-10T18:50:50.344 INFO:teuthology.orchestra.run.smithi047.stdout: librbd1 x86_64 2:17.2.7-1606.g5be57395.el9 ceph 2.9 M 2024-09-10T18:50:50.344 INFO:teuthology.orchestra.run.smithi047.stdout:Installing dependencies: 2024-09-10T18:50:50.344 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-base x86_64 2:17.2.7-1606.g5be57395.el9 ceph 5.0 M 2024-09-10T18:50:50.345 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-common x86_64 2:17.2.7-1606.g5be57395.el9 ceph 22 M 2024-09-10T18:50:50.345 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-selinux x86_64 2:17.2.7-1606.g5be57395.el9 ceph 25 k 2024-09-10T18:50:50.345 INFO:teuthology.orchestra.run.smithi047.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-10T18:50:50.345 INFO:teuthology.orchestra.run.smithi047.stdout: libcephfs2 x86_64 2:17.2.7-1606.g5be57395.el9 ceph 637 k 2024-09-10T18:50:50.345 INFO:teuthology.orchestra.run.smithi047.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-10T18:50:50.345 INFO:teuthology.orchestra.run.smithi047.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-10T18:50:50.345 INFO:teuthology.orchestra.run.smithi047.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-10T18:50:50.345 INFO:teuthology.orchestra.run.smithi047.stdout: libradosstriper1 x86_64 2:17.2.7-1606.g5be57395.el9 ceph 458 k 2024-09-10T18:50:50.345 INFO:teuthology.orchestra.run.smithi047.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-10T18:50:50.345 INFO:teuthology.orchestra.run.smithi047.stdout: librgw2 x86_64 2:17.2.7-1606.g5be57395.el9 ceph 4.9 M 2024-09-10T18:50:50.345 INFO:teuthology.orchestra.run.smithi047.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-10T18:50:50.345 INFO:teuthology.orchestra.run.smithi047.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-10T18:50:50.346 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ceph-argparse x86_64 2:17.2.7-1606.g5be57395.el9 ceph 45 k 2024-09-10T18:50:50.346 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ceph-common x86_64 2:17.2.7-1606.g5be57395.el9 ceph 120 k 2024-09-10T18:50:50.346 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cephfs x86_64 2:17.2.7-1606.g5be57395.el9 ceph 156 k 2024-09-10T18:50:50.346 INFO:teuthology.orchestra.run.smithi047.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-10T18:50:50.346 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rados x86_64 2:17.2.7-1606.g5be57395.el9 ceph 320 k 2024-09-10T18:50:50.346 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rbd x86_64 2:17.2.7-1606.g5be57395.el9 ceph 295 k 2024-09-10T18:50:50.346 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rgw x86_64 2:17.2.7-1606.g5be57395.el9 ceph 99 k 2024-09-10T18:50:50.346 INFO:teuthology.orchestra.run.smithi047.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-10T18:50:50.346 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:50:50.346 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction Summary 2024-09-10T18:50:50.346 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-10T18:50:50.346 INFO:teuthology.orchestra.run.smithi047.stdout:Install 22 Packages 2024-09-10T18:50:50.346 INFO:teuthology.orchestra.run.smithi047.stdout:Upgrade 2 Packages 2024-09-10T18:50:50.347 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:50:50.347 INFO:teuthology.orchestra.run.smithi047.stdout:Total download size: 58 M 2024-09-10T18:50:50.347 INFO:teuthology.orchestra.run.smithi047.stdout:Downloading Packages: 2024-09-10T18:50:51.117 INFO:teuthology.orchestra.run.smithi047.stdout:(1/24): ceph-base-17.2.7-1606.g5be57395.el9.x86 8.4 MB/s | 5.0 MB 00:00 2024-09-10T18:50:51.164 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-09-10T18:50:51.165 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-09-10T18:50:51.166 INFO:teuthology.orchestra.run.smithi149.stdout: Package Arch Version Repository Size 2024-09-10T18:50:51.166 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-09-10T18:50:51.166 INFO:teuthology.orchestra.run.smithi149.stdout:Installing: 2024-09-10T18:50:51.166 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-radosgw x86_64 2:17.2.7-1606.g5be57395.el9 ceph 14 M 2024-09-10T18:50:51.166 INFO:teuthology.orchestra.run.smithi149.stdout:Upgrading: 2024-09-10T18:50:51.166 INFO:teuthology.orchestra.run.smithi149.stdout: librados2 x86_64 2:17.2.7-1606.g5be57395.el9 ceph 3.1 M 2024-09-10T18:50:51.166 INFO:teuthology.orchestra.run.smithi149.stdout: librbd1 x86_64 2:17.2.7-1606.g5be57395.el9 ceph 2.9 M 2024-09-10T18:50:51.166 INFO:teuthology.orchestra.run.smithi149.stdout:Installing dependencies: 2024-09-10T18:50:51.166 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-base x86_64 2:17.2.7-1606.g5be57395.el9 ceph 5.0 M 2024-09-10T18:50:51.166 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-common x86_64 2:17.2.7-1606.g5be57395.el9 ceph 22 M 2024-09-10T18:50:51.166 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-selinux x86_64 2:17.2.7-1606.g5be57395.el9 ceph 25 k 2024-09-10T18:50:51.166 INFO:teuthology.orchestra.run.smithi149.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-10T18:50:51.166 INFO:teuthology.orchestra.run.smithi149.stdout: libcephfs2 x86_64 2:17.2.7-1606.g5be57395.el9 ceph 637 k 2024-09-10T18:50:51.167 INFO:teuthology.orchestra.run.smithi149.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-10T18:50:51.167 INFO:teuthology.orchestra.run.smithi149.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-10T18:50:51.167 INFO:teuthology.orchestra.run.smithi149.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-10T18:50:51.167 INFO:teuthology.orchestra.run.smithi149.stdout: libradosstriper1 x86_64 2:17.2.7-1606.g5be57395.el9 ceph 458 k 2024-09-10T18:50:51.167 INFO:teuthology.orchestra.run.smithi149.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-10T18:50:51.167 INFO:teuthology.orchestra.run.smithi149.stdout: librgw2 x86_64 2:17.2.7-1606.g5be57395.el9 ceph 4.9 M 2024-09-10T18:50:51.167 INFO:teuthology.orchestra.run.smithi149.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-10T18:50:51.167 INFO:teuthology.orchestra.run.smithi149.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-10T18:50:51.167 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ceph-argparse x86_64 2:17.2.7-1606.g5be57395.el9 ceph 45 k 2024-09-10T18:50:51.167 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ceph-common x86_64 2:17.2.7-1606.g5be57395.el9 ceph 120 k 2024-09-10T18:50:51.167 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cephfs x86_64 2:17.2.7-1606.g5be57395.el9 ceph 156 k 2024-09-10T18:50:51.167 INFO:teuthology.orchestra.run.smithi149.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-10T18:50:51.167 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rados x86_64 2:17.2.7-1606.g5be57395.el9 ceph 320 k 2024-09-10T18:50:51.167 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rbd x86_64 2:17.2.7-1606.g5be57395.el9 ceph 295 k 2024-09-10T18:50:51.168 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rgw x86_64 2:17.2.7-1606.g5be57395.el9 ceph 99 k 2024-09-10T18:50:51.168 INFO:teuthology.orchestra.run.smithi149.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-10T18:50:51.168 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T18:50:51.168 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction Summary 2024-09-10T18:50:51.168 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-09-10T18:50:51.168 INFO:teuthology.orchestra.run.smithi149.stdout:Install 22 Packages 2024-09-10T18:50:51.168 INFO:teuthology.orchestra.run.smithi149.stdout:Upgrade 2 Packages 2024-09-10T18:50:51.168 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T18:50:51.169 INFO:teuthology.orchestra.run.smithi149.stdout:Total download size: 58 M 2024-09-10T18:50:51.169 INFO:teuthology.orchestra.run.smithi149.stdout:Downloading Packages: 2024-09-10T18:50:51.175 INFO:teuthology.orchestra.run.smithi047.stdout:(2/24): ceph-selinux-17.2.7-1606.g5be57395.el9. 433 kB/s | 25 kB 00:00 2024-09-10T18:50:51.267 INFO:teuthology.orchestra.run.smithi047.stdout:(3/24): libcephfs2-17.2.7-1606.g5be57395.el9.x8 6.8 MB/s | 637 kB 00:00 2024-09-10T18:50:51.351 INFO:teuthology.orchestra.run.smithi047.stdout:(4/24): libradosstriper1-17.2.7-1606.g5be57395. 5.4 MB/s | 458 kB 00:00 2024-09-10T18:50:51.593 INFO:teuthology.orchestra.run.smithi047.stdout:(5/24): ceph-radosgw-17.2.7-1606.g5be57395.el9. 14 MB/s | 14 MB 00:01 2024-09-10T18:50:51.819 INFO:teuthology.orchestra.run.smithi047.stdout:(6/24): ceph-common-17.2.7-1606.g5be57395.el9.x 17 MB/s | 22 MB 00:01 2024-09-10T18:50:51.844 INFO:teuthology.orchestra.run.smithi047.stdout:(7/24): python3-ceph-argparse-17.2.7-1606.g5be5 180 kB/s | 45 kB 00:00 2024-09-10T18:50:51.906 INFO:teuthology.orchestra.run.smithi047.stdout:(8/24): python3-ceph-common-17.2.7-1606.g5be573 2.3 MB/s | 120 kB 00:00 2024-09-10T18:50:51.906 INFO:teuthology.orchestra.run.smithi047.stdout:(9/24): python3-cephfs-17.2.7-1606.g5be57395.el 2.6 MB/s | 156 kB 00:00 2024-09-10T18:50:51.928 INFO:teuthology.orchestra.run.smithi047.stdout:(10/24): python3-rados-17.2.7-1606.g5be57395.el 5.3 MB/s | 320 kB 00:00 2024-09-10T18:50:51.962 INFO:teuthology.orchestra.run.smithi047.stdout:(11/24): python3-rbd-17.2.7-1606.g5be57395.el9. 4.9 MB/s | 295 kB 00:00 2024-09-10T18:50:51.987 INFO:teuthology.orchestra.run.smithi047.stdout:(12/24): python3-rgw-17.2.7-1606.g5be57395.el9. 1.7 MB/s | 99 kB 00:00 2024-09-10T18:50:52.063 INFO:teuthology.orchestra.run.smithi047.stdout:(13/24): librgw2-17.2.7-1606.g5be57395.el9.x86_ 6.9 MB/s | 4.9 MB 00:00 2024-09-10T18:50:52.579 INFO:teuthology.orchestra.run.smithi149.stdout:(1/24): ceph-base-17.2.7-1606.g5be57395.el9.x86 6.9 MB/s | 5.0 MB 00:00 2024-09-10T18:50:52.621 INFO:teuthology.orchestra.run.smithi047.stdout:(14/24): librabbitmq-0.11.0-7.el9.x86_64.rpm 71 kB/s | 45 kB 00:00 2024-09-10T18:50:52.629 INFO:teuthology.orchestra.run.smithi149.stdout:(2/24): ceph-selinux-17.2.7-1606.g5be57395.el9. 507 kB/s | 25 kB 00:00 2024-09-10T18:50:52.737 INFO:teuthology.orchestra.run.smithi149.stdout:(3/24): libcephfs2-17.2.7-1606.g5be57395.el9.x8 5.8 MB/s | 637 kB 00:00 2024-09-10T18:50:52.771 INFO:teuthology.orchestra.run.smithi047.stdout:(15/24): libpmemobj-1.12.1-1.el9.x86_64.rpm 198 kB/s | 160 kB 00:00 2024-09-10T18:50:52.888 INFO:teuthology.orchestra.run.smithi149.stdout:(4/24): ceph-radosgw-17.2.7-1606.g5be57395.el9. 14 MB/s | 14 MB 00:01 2024-09-10T18:50:52.921 INFO:teuthology.orchestra.run.smithi149.stdout:(5/24): libradosstriper1-17.2.7-1606.g5be57395. 2.4 MB/s | 458 kB 00:00 2024-09-10T18:50:52.980 INFO:teuthology.orchestra.run.smithi149.stdout:(6/24): python3-ceph-argparse-17.2.7-1606.g5be5 773 kB/s | 45 kB 00:00 2024-09-10T18:50:53.047 INFO:teuthology.orchestra.run.smithi149.stdout:(7/24): python3-ceph-common-17.2.7-1606.g5be573 1.8 MB/s | 120 kB 00:00 2024-09-10T18:50:53.123 INFO:teuthology.orchestra.run.smithi149.stdout:(8/24): librgw2-17.2.7-1606.g5be57395.el9.x86_6 21 MB/s | 4.9 MB 00:00 2024-09-10T18:50:53.148 INFO:teuthology.orchestra.run.smithi149.stdout:(9/24): python3-cephfs-17.2.7-1606.g5be57395.el 1.5 MB/s | 156 kB 00:00 2024-09-10T18:50:53.173 INFO:teuthology.orchestra.run.smithi149.stdout:(10/24): python3-rados-17.2.7-1606.g5be57395.el 6.2 MB/s | 320 kB 00:00 2024-09-10T18:50:53.355 INFO:teuthology.orchestra.run.smithi047.stdout:(16/24): python3-prettytable-0.7.2-27.el9.noarc 72 kB/s | 42 kB 00:00 2024-09-10T18:50:53.405 INFO:teuthology.orchestra.run.smithi047.stdout:(17/24): gperftools-libs-2.9.1-2.el9.x86_64.rpm 6.1 MB/s | 309 kB 00:00 2024-09-10T18:50:53.407 INFO:teuthology.orchestra.run.smithi149.stdout:(11/24): ceph-common-17.2.7-1606.g5be57395.el9. 15 MB/s | 22 MB 00:01 2024-09-10T18:50:53.432 INFO:teuthology.orchestra.run.smithi149.stdout:(12/24): python3-rgw-17.2.7-1606.g5be57395.el9. 384 kB/s | 99 kB 00:00 2024-09-10T18:50:53.439 INFO:teuthology.orchestra.run.smithi047.stdout:(18/24): librdkafka-1.6.1-102.el9.x86_64.rpm 481 kB/s | 662 kB 00:01 2024-09-10T18:50:53.456 INFO:teuthology.orchestra.run.smithi047.stdout:(19/24): liboath-2.6.7-2.el9.x86_64.rpm 970 kB/s | 49 kB 00:00 2024-09-10T18:50:53.465 INFO:teuthology.orchestra.run.smithi149.stdout:(13/24): python3-rbd-17.2.7-1606.g5be57395.el9. 928 kB/s | 295 kB 00:00 2024-09-10T18:50:53.531 INFO:teuthology.orchestra.run.smithi047.stdout:(20/24): libunwind-1.6.2-1.el9.x86_64.rpm 733 kB/s | 67 kB 00:00 2024-09-10T18:50:53.556 INFO:teuthology.orchestra.run.smithi047.stdout:(21/24): lttng-ust-2.12.0-6.el9.x86_64.rpm 313 kB/s | 292 kB 00:00 2024-09-10T18:50:53.591 INFO:teuthology.orchestra.run.smithi149.stdout:(14/24): librabbitmq-0.11.0-7.el9.x86_64.rpm 285 kB/s | 45 kB 00:00 2024-09-10T18:50:53.606 INFO:teuthology.orchestra.run.smithi047.stdout:(22/24): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-09-10T18:50:53.624 INFO:teuthology.orchestra.run.smithi149.stdout:(15/24): libpmemobj-1.12.1-1.el9.x86_64.rpm 738 kB/s | 160 kB 00:00 2024-09-10T18:50:53.683 INFO:teuthology.orchestra.run.smithi149.stdout:(16/24): python3-prettytable-0.7.2-27.el9.noarc 725 kB/s | 42 kB 00:00 2024-09-10T18:50:53.717 INFO:teuthology.orchestra.run.smithi149.stdout:(17/24): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.3 MB/s | 292 kB 00:00 2024-09-10T18:50:53.759 INFO:teuthology.orchestra.run.smithi149.stdout:(18/24): gperftools-libs-2.9.1-2.el9.x86_64.rpm 4.0 MB/s | 309 kB 00:00 2024-09-10T18:50:53.782 INFO:teuthology.orchestra.run.smithi047.stdout:(23/24): librados2-17.2.7-1606.g5be57395.el9.x8 12 MB/s | 3.1 MB 00:00 2024-09-10T18:50:53.784 INFO:teuthology.orchestra.run.smithi149.stdout:(19/24): libunwind-1.6.2-1.el9.x86_64.rpm 2.8 MB/s | 67 kB 00:00 2024-09-10T18:50:53.809 INFO:teuthology.orchestra.run.smithi149.stdout:(20/24): liboath-2.6.7-2.el9.x86_64.rpm 538 kB/s | 49 kB 00:00 2024-09-10T18:50:53.857 INFO:teuthology.orchestra.run.smithi047.stdout:(24/24): librbd1-17.2.7-1606.g5be57395.el9.x86_ 9.8 MB/s | 2.9 MB 00:00 2024-09-10T18:50:53.859 INFO:teuthology.orchestra.run.smithi149.stdout:(21/24): thrift-0.15.0-2.el9.x86_64.rpm 21 MB/s | 1.6 MB 00:00 2024-09-10T18:50:53.862 INFO:teuthology.orchestra.run.smithi047.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:50:53.862 INFO:teuthology.orchestra.run.smithi047.stdout:Total 17 MB/s | 58 MB 00:03 2024-09-10T18:50:53.893 INFO:teuthology.orchestra.run.smithi149.stdout:(22/24): librdkafka-1.6.1-102.el9.x86_64.rpm 1.5 MB/s | 662 kB 00:00 2024-09-10T18:50:53.931 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction check 2024-09-10T18:50:53.970 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction check succeeded. 2024-09-10T18:50:53.970 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction test 2024-09-10T18:50:54.010 INFO:teuthology.orchestra.run.smithi149.stdout:(23/24): librados2-17.2.7-1606.g5be57395.el9.x8 15 MB/s | 3.1 MB 00:00 2024-09-10T18:50:54.060 INFO:teuthology.orchestra.run.smithi149.stdout:(24/24): librbd1-17.2.7-1606.g5be57395.el9.x86_ 15 MB/s | 2.9 MB 00:00 2024-09-10T18:50:54.064 INFO:teuthology.orchestra.run.smithi149.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:50:54.064 INFO:teuthology.orchestra.run.smithi149.stdout:Total 20 MB/s | 58 MB 00:02 2024-09-10T18:50:54.126 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction check 2024-09-10T18:50:54.159 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction check succeeded. 2024-09-10T18:50:54.159 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction test 2024-09-10T18:50:54.472 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction test succeeded. 2024-09-10T18:50:54.473 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction 2024-09-10T18:50:54.650 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction test succeeded. 2024-09-10T18:50:54.650 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction 2024-09-10T18:50:55.680 INFO:teuthology.orchestra.run.smithi047.stdout: Preparing : 1/1 2024-09-10T18:50:55.835 INFO:teuthology.orchestra.run.smithi149.stdout: Preparing : 1/1 2024-09-10T18:50:56.088 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/26 2024-09-10T18:50:56.151 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : librados2-2:17.2.7-1606.g5be57395.el9.x86_64 2/26 2024-09-10T18:50:56.217 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/26 2024-09-10T18:50:56.252 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: librados2-2:17.2.7-1606.g5be57395.el9.x86_64 2/26 2024-09-10T18:50:56.287 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : librados2-2:17.2.7-1606.g5be57395.el9.x86_64 2/26 2024-09-10T18:50:56.333 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-rados-2:17.2.7-1606.g5be57395.el9.x86_64 3/26 2024-09-10T18:50:56.361 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libcephfs2-2:17.2.7-1606.g5be57395.el9.x86_64 4/26 2024-09-10T18:50:56.380 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: librados2-2:17.2.7-1606.g5be57395.el9.x86_64 2/26 2024-09-10T18:50:56.434 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: libcephfs2-2:17.2.7-1606.g5be57395.el9.x86_64 4/26 2024-09-10T18:50:56.459 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-rados-2:17.2.7-1606.g5be57395.el9.x86_64 3/26 2024-09-10T18:50:56.489 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libcephfs2-2:17.2.7-1606.g5be57395.el9.x86_64 4/26 2024-09-10T18:50:56.491 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : thrift-0.15.0-2.el9.x86_64 5/26 2024-09-10T18:50:56.524 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/26 2024-09-10T18:50:56.576 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: libcephfs2-2:17.2.7-1606.g5be57395.el9.x86_64 4/26 2024-09-10T18:50:56.624 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : thrift-0.15.0-2.el9.x86_64 5/26 2024-09-10T18:50:56.661 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/26 2024-09-10T18:50:56.993 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/26 2024-09-10T18:50:57.022 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : librgw2-2:17.2.7-1606.g5be57395.el9.x86_64 8/26 2024-09-10T18:50:57.085 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: librgw2-2:17.2.7-1606.g5be57395.el9.x86_64 8/26 2024-09-10T18:50:57.103 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/26 2024-09-10T18:50:57.122 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/26 2024-09-10T18:50:57.134 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : librgw2-2:17.2.7-1606.g5be57395.el9.x86_64 8/26 2024-09-10T18:50:57.174 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-ceph-argparse-2:17.2.7-1606.g5be57395.el9. 10/26 2024-09-10T18:50:57.197 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: librgw2-2:17.2.7-1606.g5be57395.el9.x86_64 8/26 2024-09-10T18:50:57.211 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cephfs-2:17.2.7-1606.g5be57395.el9.x86_64 11/26 2024-09-10T18:50:57.242 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/26 2024-09-10T18:50:57.274 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-rgw-2:17.2.7-1606.g5be57395.el9.x86_64 12/26 2024-09-10T18:50:57.285 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-ceph-argparse-2:17.2.7-1606.g5be57395.el9. 10/26 2024-09-10T18:50:57.300 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libradosstriper1-2:17.2.7-1606.g5be57395.el9.x86_6 13/26 2024-09-10T18:50:57.322 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cephfs-2:17.2.7-1606.g5be57395.el9.x86_64 11/26 2024-09-10T18:50:57.366 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: libradosstriper1-2:17.2.7-1606.g5be57395.el9.x86_6 13/26 2024-09-10T18:50:57.384 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-rgw-2:17.2.7-1606.g5be57395.el9.x86_64 12/26 2024-09-10T18:50:57.412 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/26 2024-09-10T18:50:57.412 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libradosstriper1-2:17.2.7-1606.g5be57395.el9.x86_6 13/26 2024-09-10T18:50:57.450 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/26 2024-09-10T18:50:57.461 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: libradosstriper1-2:17.2.7-1606.g5be57395.el9.x86_6 13/26 2024-09-10T18:50:57.483 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 16/26 2024-09-10T18:50:57.501 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/26 2024-09-10T18:50:57.537 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/26 2024-09-10T18:50:57.570 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 16/26 2024-09-10T18:50:57.802 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/26 2024-09-10T18:50:57.838 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : librbd1-2:17.2.7-1606.g5be57395.el9.x86_64 18/26 2024-09-10T18:50:57.867 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/26 2024-09-10T18:50:57.908 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : librbd1-2:17.2.7-1606.g5be57395.el9.x86_64 18/26 2024-09-10T18:50:57.913 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: librbd1-2:17.2.7-1606.g5be57395.el9.x86_64 18/26 2024-09-10T18:50:57.989 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-rbd-2:17.2.7-1606.g5be57395.el9.x86_64 19/26 2024-09-10T18:50:57.998 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: librbd1-2:17.2.7-1606.g5be57395.el9.x86_64 18/26 2024-09-10T18:50:58.016 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-ceph-common-2:17.2.7-1606.g5be57395.el9.x8 20/26 2024-09-10T18:50:58.072 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-rbd-2:17.2.7-1606.g5be57395.el9.x86_64 19/26 2024-09-10T18:50:58.101 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-ceph-common-2:17.2.7-1606.g5be57395.el9.x8 20/26 2024-09-10T18:51:00.580 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-common-2:17.2.7-1606.g5be57395.el9.x86_64 21/26 2024-09-10T18:51:00.612 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-common-2:17.2.7-1606.g5be57395.el9.x86_64 21/26 2024-09-10T18:51:00.626 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-common-2:17.2.7-1606.g5be57395.el9.x86_64 21/26 2024-09-10T18:51:00.654 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-common-2:17.2.7-1606.g5be57395.el9.x86_64 21/26 2024-09-10T18:51:01.177 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-common-2:17.2.7-1606.g5be57395.el9.x86_64 21/26 2024-09-10T18:51:01.212 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-base-2:17.2.7-1606.g5be57395.el9.x86_64 22/26 2024-09-10T18:51:01.222 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-common-2:17.2.7-1606.g5be57395.el9.x86_64 21/26 2024-09-10T18:51:01.258 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-base-2:17.2.7-1606.g5be57395.el9.x86_64 22/26 2024-09-10T18:51:01.295 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-base-2:17.2.7-1606.g5be57395.el9.x86_64 22/26 2024-09-10T18:51:01.296 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-10T18:51:01.296 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-10T18:51:01.296 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T18:51:01.331 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-selinux-2:17.2.7-1606.g5be57395.el9.x86_64 23/26 2024-09-10T18:51:01.366 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-base-2:17.2.7-1606.g5be57395.el9.x86_64 22/26 2024-09-10T18:51:01.366 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-10T18:51:01.366 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-10T18:51:01.366 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:51:01.400 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-selinux-2:17.2.7-1606.g5be57395.el9.x86_64 23/26 2024-09-10T18:51:11.100 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-selinux-2:17.2.7-1606.g5be57395.el9.x86_64 23/26 2024-09-10T18:51:11.101 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /sys 2024-09-10T18:51:11.101 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /proc 2024-09-10T18:51:11.101 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /mnt 2024-09-10T18:51:11.101 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /var/tmp 2024-09-10T18:51:11.101 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /home 2024-09-10T18:51:11.101 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /root 2024-09-10T18:51:11.101 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /tmp 2024-09-10T18:51:11.101 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T18:51:11.459 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-selinux-2:17.2.7-1606.g5be57395.el9.x86_64 23/26 2024-09-10T18:51:11.459 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /sys 2024-09-10T18:51:11.459 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /proc 2024-09-10T18:51:11.459 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /mnt 2024-09-10T18:51:11.459 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /var/tmp 2024-09-10T18:51:11.459 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /home 2024-09-10T18:51:11.459 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /root 2024-09-10T18:51:11.459 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /tmp 2024-09-10T18:51:11.460 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:51:12.404 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-radosgw-2:17.2.7-1606.g5be57395.el9.x86_64 24/26 2024-09-10T18:51:12.484 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-1606.g5be57395.el9.x86_64 24/26 2024-09-10T18:51:12.484 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T18:51:12.484 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-10T18:51:12.484 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T18:51:12.485 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T18:51:12.485 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T18:51:12.486 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-09-10T18:51:12.574 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-09-10T18:51:12.574 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 26/26 2024-09-10T18:51:12.818 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-radosgw-2:17.2.7-1606.g5be57395.el9.x86_64 24/26 2024-09-10T18:51:12.898 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-1606.g5be57395.el9.x86_64 24/26 2024-09-10T18:51:12.898 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T18:51:12.898 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-10T18:51:12.899 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T18:51:12.899 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T18:51:12.899 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:51:12.900 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-09-10T18:51:13.004 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-09-10T18:51:13.005 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 26/26 2024-09-10T18:51:13.797 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 26/26 2024-09-10T18:51:13.797 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-base-2:17.2.7-1606.g5be57395.el9.x86_64 1/26 2024-09-10T18:51:13.797 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-common-2:17.2.7-1606.g5be57395.el9.x86_64 2/26 2024-09-10T18:51:13.797 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-radosgw-2:17.2.7-1606.g5be57395.el9.x86_64 3/26 2024-09-10T18:51:13.797 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-selinux-2:17.2.7-1606.g5be57395.el9.x86_64 4/26 2024-09-10T18:51:13.797 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libcephfs2-2:17.2.7-1606.g5be57395.el9.x86_64 5/26 2024-09-10T18:51:13.797 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libradosstriper1-2:17.2.7-1606.g5be57395.el9.x86_6 6/26 2024-09-10T18:51:13.797 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librgw2-2:17.2.7-1606.g5be57395.el9.x86_64 7/26 2024-09-10T18:51:13.798 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-ceph-argparse-2:17.2.7-1606.g5be57395.el9. 8/26 2024-09-10T18:51:13.798 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-ceph-common-2:17.2.7-1606.g5be57395.el9.x8 9/26 2024-09-10T18:51:13.798 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cephfs-2:17.2.7-1606.g5be57395.el9.x86_64 10/26 2024-09-10T18:51:13.798 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-rados-2:17.2.7-1606.g5be57395.el9.x86_64 11/26 2024-09-10T18:51:13.798 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-rbd-2:17.2.7-1606.g5be57395.el9.x86_64 12/26 2024-09-10T18:51:13.798 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-rgw-2:17.2.7-1606.g5be57395.el9.x86_64 13/26 2024-09-10T18:51:13.798 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 14/26 2024-09-10T18:51:13.798 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 15/26 2024-09-10T18:51:13.798 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/26 2024-09-10T18:51:13.798 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/26 2024-09-10T18:51:13.798 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/26 2024-09-10T18:51:13.798 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 19/26 2024-09-10T18:51:13.798 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 20/26 2024-09-10T18:51:13.799 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 21/26 2024-09-10T18:51:13.799 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 22/26 2024-09-10T18:51:13.799 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librados2-2:17.2.7-1606.g5be57395.el9.x86_64 23/26 2024-09-10T18:51:13.799 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 24/26 2024-09-10T18:51:13.799 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librbd1-2:17.2.7-1606.g5be57395.el9.x86_64 25/26 2024-09-10T18:51:14.282 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 26/26 2024-09-10T18:51:14.282 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-base-2:17.2.7-1606.g5be57395.el9.x86_64 1/26 2024-09-10T18:51:14.282 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-common-2:17.2.7-1606.g5be57395.el9.x86_64 2/26 2024-09-10T18:51:14.282 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-radosgw-2:17.2.7-1606.g5be57395.el9.x86_64 3/26 2024-09-10T18:51:14.282 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-selinux-2:17.2.7-1606.g5be57395.el9.x86_64 4/26 2024-09-10T18:51:14.283 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libcephfs2-2:17.2.7-1606.g5be57395.el9.x86_64 5/26 2024-09-10T18:51:14.283 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libradosstriper1-2:17.2.7-1606.g5be57395.el9.x86_6 6/26 2024-09-10T18:51:14.283 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librgw2-2:17.2.7-1606.g5be57395.el9.x86_64 7/26 2024-09-10T18:51:14.283 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-ceph-argparse-2:17.2.7-1606.g5be57395.el9. 8/26 2024-09-10T18:51:14.283 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-ceph-common-2:17.2.7-1606.g5be57395.el9.x8 9/26 2024-09-10T18:51:14.283 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cephfs-2:17.2.7-1606.g5be57395.el9.x86_64 10/26 2024-09-10T18:51:14.283 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-rados-2:17.2.7-1606.g5be57395.el9.x86_64 11/26 2024-09-10T18:51:14.284 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-rbd-2:17.2.7-1606.g5be57395.el9.x86_64 12/26 2024-09-10T18:51:14.284 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-rgw-2:17.2.7-1606.g5be57395.el9.x86_64 13/26 2024-09-10T18:51:14.284 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 14/26 2024-09-10T18:51:14.284 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 15/26 2024-09-10T18:51:14.284 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/26 2024-09-10T18:51:14.284 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/26 2024-09-10T18:51:14.284 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/26 2024-09-10T18:51:14.284 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 19/26 2024-09-10T18:51:14.284 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 20/26 2024-09-10T18:51:14.284 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 21/26 2024-09-10T18:51:14.284 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 22/26 2024-09-10T18:51:14.284 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librados2-2:17.2.7-1606.g5be57395.el9.x86_64 23/26 2024-09-10T18:51:14.284 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 24/26 2024-09-10T18:51:14.285 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librbd1-2:17.2.7-1606.g5be57395.el9.x86_64 25/26 2024-09-10T18:51:14.698 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 26/26 2024-09-10T18:51:14.698 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T18:51:14.698 INFO:teuthology.orchestra.run.smithi149.stdout:Upgraded: 2024-09-10T18:51:14.698 INFO:teuthology.orchestra.run.smithi149.stdout: librados2-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:51:14.698 INFO:teuthology.orchestra.run.smithi149.stdout: librbd1-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:51:14.698 INFO:teuthology.orchestra.run.smithi149.stdout:Installed: 2024-09-10T18:51:14.698 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-base-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:51:14.698 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-common-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:51:14.698 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-radosgw-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:51:14.699 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-selinux-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:51:14.699 INFO:teuthology.orchestra.run.smithi149.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-10T18:51:14.699 INFO:teuthology.orchestra.run.smithi149.stdout: libcephfs2-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:51:14.699 INFO:teuthology.orchestra.run.smithi149.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-10T18:51:14.699 INFO:teuthology.orchestra.run.smithi149.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-10T18:51:14.699 INFO:teuthology.orchestra.run.smithi149.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-10T18:51:14.699 INFO:teuthology.orchestra.run.smithi149.stdout: libradosstriper1-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:51:14.699 INFO:teuthology.orchestra.run.smithi149.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-10T18:51:14.699 INFO:teuthology.orchestra.run.smithi149.stdout: librgw2-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:51:14.699 INFO:teuthology.orchestra.run.smithi149.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-10T18:51:14.699 INFO:teuthology.orchestra.run.smithi149.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-10T18:51:14.699 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ceph-argparse-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:51:14.699 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ceph-common-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:51:14.700 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cephfs-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:51:14.700 INFO:teuthology.orchestra.run.smithi149.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-10T18:51:14.700 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rados-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:51:14.700 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rbd-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:51:14.700 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rgw-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:51:14.700 INFO:teuthology.orchestra.run.smithi149.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-10T18:51:14.700 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T18:51:14.700 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-09-10T18:51:15.052 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install ceph-test 2024-09-10T18:51:15.281 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 26/26 2024-09-10T18:51:15.281 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:51:15.281 INFO:teuthology.orchestra.run.smithi047.stdout:Upgraded: 2024-09-10T18:51:15.282 INFO:teuthology.orchestra.run.smithi047.stdout: librados2-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:51:15.282 INFO:teuthology.orchestra.run.smithi047.stdout: librbd1-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:51:15.282 INFO:teuthology.orchestra.run.smithi047.stdout:Installed: 2024-09-10T18:51:15.282 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-base-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:51:15.282 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-common-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:51:15.282 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-radosgw-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:51:15.282 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-selinux-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:51:15.282 INFO:teuthology.orchestra.run.smithi047.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-10T18:51:15.282 INFO:teuthology.orchestra.run.smithi047.stdout: libcephfs2-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:51:15.282 INFO:teuthology.orchestra.run.smithi047.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-10T18:51:15.282 INFO:teuthology.orchestra.run.smithi047.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-10T18:51:15.282 INFO:teuthology.orchestra.run.smithi047.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-10T18:51:15.283 INFO:teuthology.orchestra.run.smithi047.stdout: libradosstriper1-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:51:15.283 INFO:teuthology.orchestra.run.smithi047.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-10T18:51:15.283 INFO:teuthology.orchestra.run.smithi047.stdout: librgw2-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:51:15.283 INFO:teuthology.orchestra.run.smithi047.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-10T18:51:15.283 INFO:teuthology.orchestra.run.smithi047.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-10T18:51:15.283 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ceph-argparse-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:51:15.283 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ceph-common-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:51:15.283 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cephfs-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:51:15.283 INFO:teuthology.orchestra.run.smithi047.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-10T18:51:15.283 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rados-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:51:15.283 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rbd-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:51:15.283 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rgw-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:51:15.283 INFO:teuthology.orchestra.run.smithi047.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-10T18:51:15.284 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:51:15.284 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-09-10T18:51:15.629 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:00:27 ago on Tue 10 Sep 2024 06:50:48 PM UTC. 2024-09-10T18:51:15.666 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install ceph-test 2024-09-10T18:51:15.763 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-09-10T18:51:15.764 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-09-10T18:51:15.764 INFO:teuthology.orchestra.run.smithi149.stdout: Package Arch Version Repository Size 2024-09-10T18:51:15.764 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-09-10T18:51:15.764 INFO:teuthology.orchestra.run.smithi149.stdout:Installing: 2024-09-10T18:51:15.764 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-test x86_64 2:17.2.7-1606.g5be57395.el9 ceph 48 M 2024-09-10T18:51:15.764 INFO:teuthology.orchestra.run.smithi149.stdout:Installing dependencies: 2024-09-10T18:51:15.764 INFO:teuthology.orchestra.run.smithi149.stdout: libcephsqlite x86_64 2:17.2.7-1606.g5be57395.el9 ceph 163 k 2024-09-10T18:51:15.765 INFO:teuthology.orchestra.run.smithi149.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-10T18:51:15.765 INFO:teuthology.orchestra.run.smithi149.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-10T18:51:15.765 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T18:51:15.765 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction Summary 2024-09-10T18:51:15.765 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-09-10T18:51:15.765 INFO:teuthology.orchestra.run.smithi149.stdout:Install 4 Packages 2024-09-10T18:51:15.765 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T18:51:15.766 INFO:teuthology.orchestra.run.smithi149.stdout:Total download size: 48 M 2024-09-10T18:51:15.766 INFO:teuthology.orchestra.run.smithi149.stdout:Installed size: 193 M 2024-09-10T18:51:15.766 INFO:teuthology.orchestra.run.smithi149.stdout:Downloading Packages: 2024-09-10T18:51:16.222 INFO:teuthology.orchestra.run.smithi149.stdout:(1/4): libcephsqlite-17.2.7-1606.g5be57395.el9. 793 kB/s | 163 kB 00:00 2024-09-10T18:51:16.252 INFO:teuthology.orchestra.run.smithi047.stdout:Last metadata expiration check: 0:00:29 ago on Tue 10 Sep 2024 06:50:47 PM UTC. 2024-09-10T18:51:16.409 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-09-10T18:51:16.409 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-10T18:51:16.410 INFO:teuthology.orchestra.run.smithi047.stdout: Package Arch Version Repository Size 2024-09-10T18:51:16.410 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-10T18:51:16.410 INFO:teuthology.orchestra.run.smithi047.stdout:Installing: 2024-09-10T18:51:16.410 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-test x86_64 2:17.2.7-1606.g5be57395.el9 ceph 48 M 2024-09-10T18:51:16.410 INFO:teuthology.orchestra.run.smithi047.stdout:Installing dependencies: 2024-09-10T18:51:16.410 INFO:teuthology.orchestra.run.smithi047.stdout: libcephsqlite x86_64 2:17.2.7-1606.g5be57395.el9 ceph 163 k 2024-09-10T18:51:16.410 INFO:teuthology.orchestra.run.smithi047.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-10T18:51:16.410 INFO:teuthology.orchestra.run.smithi047.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-10T18:51:16.410 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:51:16.410 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction Summary 2024-09-10T18:51:16.410 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-10T18:51:16.410 INFO:teuthology.orchestra.run.smithi047.stdout:Install 4 Packages 2024-09-10T18:51:16.410 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:51:16.411 INFO:teuthology.orchestra.run.smithi047.stdout:Total download size: 48 M 2024-09-10T18:51:16.411 INFO:teuthology.orchestra.run.smithi047.stdout:Installed size: 193 M 2024-09-10T18:51:16.411 INFO:teuthology.orchestra.run.smithi047.stdout:Downloading Packages: 2024-09-10T18:51:16.831 INFO:teuthology.orchestra.run.smithi149.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 105 kB/s | 64 kB 00:00 2024-09-10T18:51:16.850 INFO:teuthology.orchestra.run.smithi047.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 1.6 MB/s | 304 kB 00:00 2024-09-10T18:51:16.900 INFO:teuthology.orchestra.run.smithi047.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-10T18:51:17.006 INFO:teuthology.orchestra.run.smithi149.stdout:(3/4): socat-1.7.4.1-6.el9.x86_64.rpm 307 kB/s | 304 kB 00:00 2024-09-10T18:51:17.101 INFO:teuthology.orchestra.run.smithi047.stdout:(3/4): libcephsqlite-17.2.7-1606.g5be57395.el9. 375 kB/s | 163 kB 00:00 2024-09-10T18:51:17.490 INFO:teuthology.orchestra.run.smithi149.stdout:(4/4): ceph-test-17.2.7-1606.g5be57395.el9.x86_ 33 MB/s | 48 MB 00:01 2024-09-10T18:51:17.492 INFO:teuthology.orchestra.run.smithi149.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:51:17.492 INFO:teuthology.orchestra.run.smithi149.stdout:Total 28 MB/s | 48 MB 00:01 2024-09-10T18:51:17.521 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction check 2024-09-10T18:51:17.540 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction check succeeded. 2024-09-10T18:51:17.540 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction test 2024-09-10T18:51:17.909 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction test succeeded. 2024-09-10T18:51:17.909 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction 2024-09-10T18:51:18.430 INFO:teuthology.orchestra.run.smithi149.stdout: Preparing : 1/1 2024-09-10T18:51:18.536 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-10T18:51:18.588 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-10T18:51:18.609 INFO:teuthology.orchestra.run.smithi047.stdout:(4/4): ceph-test-17.2.7-1606.g5be57395.el9.x86_ 25 MB/s | 48 MB 00:01 2024-09-10T18:51:18.611 INFO:teuthology.orchestra.run.smithi047.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:51:18.612 INFO:teuthology.orchestra.run.smithi047.stdout:Total 22 MB/s | 48 MB 00:02 2024-09-10T18:51:18.616 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libcephsqlite-2:17.2.7-1606.g5be57395.el9.x86_64 3/4 2024-09-10T18:51:18.640 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction check 2024-09-10T18:51:18.661 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction check succeeded. 2024-09-10T18:51:18.661 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction test 2024-09-10T18:51:19.048 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction test succeeded. 2024-09-10T18:51:19.049 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction 2024-09-10T18:51:19.598 INFO:teuthology.orchestra.run.smithi047.stdout: Preparing : 1/1 2024-09-10T18:51:19.706 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-10T18:51:19.759 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-10T18:51:19.786 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libcephsqlite-2:17.2.7-1606.g5be57395.el9.x86_64 3/4 2024-09-10T18:51:23.071 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: libcephsqlite-2:17.2.7-1606.g5be57395.el9.x86_64 3/4 2024-09-10T18:51:23.148 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-test-2:17.2.7-1606.g5be57395.el9.x86_64 4/4 2024-09-10T18:51:24.451 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: libcephsqlite-2:17.2.7-1606.g5be57395.el9.x86_64 3/4 2024-09-10T18:51:24.541 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-test-2:17.2.7-1606.g5be57395.el9.x86_64 4/4 2024-09-10T18:51:25.402 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-test-2:17.2.7-1606.g5be57395.el9.x86_64 4/4 2024-09-10T18:51:25.402 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-test-2:17.2.7-1606.g5be57395.el9.x86_64 1/4 2024-09-10T18:51:25.402 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libcephsqlite-2:17.2.7-1606.g5be57395.el9.x86_64 2/4 2024-09-10T18:51:25.402 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-10T18:51:25.718 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-10T18:51:25.718 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T18:51:25.718 INFO:teuthology.orchestra.run.smithi149.stdout:Installed: 2024-09-10T18:51:25.718 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-test-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:51:25.718 INFO:teuthology.orchestra.run.smithi149.stdout: libcephsqlite-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:51:25.718 INFO:teuthology.orchestra.run.smithi149.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-10T18:51:25.718 INFO:teuthology.orchestra.run.smithi149.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-10T18:51:25.718 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T18:51:25.718 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-09-10T18:51:26.009 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install ceph 2024-09-10T18:51:26.583 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:00:38 ago on Tue 10 Sep 2024 06:50:48 PM UTC. 2024-09-10T18:51:26.719 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-09-10T18:51:26.721 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-09-10T18:51:26.722 INFO:teuthology.orchestra.run.smithi149.stdout: Package Arch Version Repository Size 2024-09-10T18:51:26.722 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-09-10T18:51:26.722 INFO:teuthology.orchestra.run.smithi149.stdout:Installing: 2024-09-10T18:51:26.722 INFO:teuthology.orchestra.run.smithi149.stdout: ceph x86_64 2:17.2.7-1606.g5be57395.el9 ceph 6.5 k 2024-09-10T18:51:26.722 INFO:teuthology.orchestra.run.smithi149.stdout:Installing dependencies: 2024-09-10T18:51:26.722 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mds x86_64 2:17.2.7-1606.g5be57395.el9 ceph 2.1 M 2024-09-10T18:51:26.722 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr x86_64 2:17.2.7-1606.g5be57395.el9 ceph 1.4 M 2024-09-10T18:51:26.722 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-modules-core noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 240 k 2024-09-10T18:51:26.722 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mon x86_64 2:17.2.7-1606.g5be57395.el9 ceph 3.9 M 2024-09-10T18:51:26.722 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-osd x86_64 2:17.2.7-1606.g5be57395.el9 ceph 15 M 2024-09-10T18:51:26.722 INFO:teuthology.orchestra.run.smithi149.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-10T18:51:26.723 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-10T18:51:26.723 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-10T18:51:26.723 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-10T18:51:26.723 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-10T18:51:26.723 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-10T18:51:26.723 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-10T18:51:26.723 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-collections 2024-09-10T18:51:26.723 INFO:teuthology.orchestra.run.smithi149.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-09-10T18:51:26.723 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-10T18:51:26.723 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-10T18:51:26.723 INFO:teuthology.orchestra.run.smithi149.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-10T18:51:26.723 INFO:teuthology.orchestra.run.smithi149.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-10T18:51:26.723 INFO:teuthology.orchestra.run.smithi149.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-10T18:51:26.724 INFO:teuthology.orchestra.run.smithi149.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-10T18:51:26.724 INFO:teuthology.orchestra.run.smithi149.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-10T18:51:26.724 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-10T18:51:26.724 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-10T18:51:26.724 INFO:teuthology.orchestra.run.smithi149.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-10T18:51:26.724 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-10T18:51:26.724 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-10T18:51:26.724 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-10T18:51:26.724 INFO:teuthology.orchestra.run.smithi149.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-10T18:51:26.724 INFO:teuthology.orchestra.run.smithi149.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-10T18:51:26.724 INFO:teuthology.orchestra.run.smithi149.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-10T18:51:26.724 INFO:teuthology.orchestra.run.smithi149.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-10T18:51:26.724 INFO:teuthology.orchestra.run.smithi149.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-10T18:51:26.725 INFO:teuthology.orchestra.run.smithi149.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-10T18:51:26.725 INFO:teuthology.orchestra.run.smithi149.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-10T18:51:26.725 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T18:51:26.725 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction Summary 2024-09-10T18:51:26.725 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-09-10T18:51:26.725 INFO:teuthology.orchestra.run.smithi149.stdout:Install 34 Packages 2024-09-10T18:51:26.725 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T18:51:26.726 INFO:teuthology.orchestra.run.smithi149.stdout:Total download size: 27 M 2024-09-10T18:51:26.726 INFO:teuthology.orchestra.run.smithi149.stdout:Installed size: 96 M 2024-09-10T18:51:26.726 INFO:teuthology.orchestra.run.smithi149.stdout:Downloading Packages: 2024-09-10T18:51:26.746 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-test-2:17.2.7-1606.g5be57395.el9.x86_64 4/4 2024-09-10T18:51:26.747 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-test-2:17.2.7-1606.g5be57395.el9.x86_64 1/4 2024-09-10T18:51:26.747 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libcephsqlite-2:17.2.7-1606.g5be57395.el9.x86_64 2/4 2024-09-10T18:51:26.747 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-10T18:51:27.053 INFO:teuthology.orchestra.run.smithi149.stdout:(1/34): ceph-17.2.7-1606.g5be57395.el9.x86_64.r 51 kB/s | 6.5 kB 00:00 2024-09-10T18:51:27.059 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-10T18:51:27.059 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:51:27.059 INFO:teuthology.orchestra.run.smithi047.stdout:Installed: 2024-09-10T18:51:27.059 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-test-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:51:27.059 INFO:teuthology.orchestra.run.smithi047.stdout: libcephsqlite-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:51:27.059 INFO:teuthology.orchestra.run.smithi047.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-10T18:51:27.060 INFO:teuthology.orchestra.run.smithi047.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-10T18:51:27.060 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:51:27.060 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-09-10T18:51:27.270 INFO:teuthology.orchestra.run.smithi149.stdout:(2/34): ceph-mds-17.2.7-1606.g5be57395.el9.x86_ 6.0 MB/s | 2.1 MB 00:00 2024-09-10T18:51:27.311 INFO:teuthology.orchestra.run.smithi149.stdout:(3/34): ceph-mgr-17.2.7-1606.g5be57395.el9.x86_ 3.6 MB/s | 1.4 MB 00:00 2024-09-10T18:51:27.374 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install ceph 2024-09-10T18:51:27.404 INFO:teuthology.orchestra.run.smithi149.stdout:(4/34): ceph-mon-17.2.7-1606.g5be57395.el9.x86_ 11 MB/s | 3.9 MB 00:00 2024-09-10T18:51:27.454 INFO:teuthology.orchestra.run.smithi149.stdout:(5/34): ceph-mgr-modules-core-17.2.7-1606.g5be5 1.7 MB/s | 240 kB 00:00 2024-09-10T18:51:27.579 INFO:teuthology.orchestra.run.smithi149.stdout:(6/34): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.4 MB/s | 253 kB 00:00 2024-09-10T18:51:27.638 INFO:teuthology.orchestra.run.smithi149.stdout:(7/34): python3-cryptography-36.0.1-4.el9.x86_6 6.7 MB/s | 1.2 MB 00:00 2024-09-10T18:51:27.788 INFO:teuthology.orchestra.run.smithi149.stdout:(8/34): ceph-osd-17.2.7-1606.g5be57395.el9.x86_ 30 MB/s | 15 MB 00:00 2024-09-10T18:51:27.813 INFO:teuthology.orchestra.run.smithi149.stdout:(9/34): python3-ply-3.11-14.el9.noarch.rpm 455 kB/s | 106 kB 00:00 2024-09-10T18:51:27.839 INFO:teuthology.orchestra.run.smithi149.stdout:(10/34): python3-pycparser-2.20-6.el9.noarch.rp 674 kB/s | 135 kB 00:00 2024-09-10T18:51:27.914 INFO:teuthology.orchestra.run.smithi149.stdout:(11/34): python3-urllib3-1.26.5-6.el9.noarch.rp 2.1 MB/s | 215 kB 00:00 2024-09-10T18:51:27.939 INFO:teuthology.orchestra.run.smithi149.stdout:(12/34): python3-requests-2.25.1-8.el9.noarch.r 828 kB/s | 125 kB 00:00 2024-09-10T18:51:27.965 INFO:teuthology.orchestra.run.smithi047.stdout:Last metadata expiration check: 0:00:40 ago on Tue 10 Sep 2024 06:50:47 PM UTC. 2024-09-10T18:51:28.023 INFO:teuthology.orchestra.run.smithi149.stdout:(13/34): python3-markupsafe-1.1.1-12.el9.x86_64 320 kB/s | 35 kB 00:00 2024-09-10T18:51:28.048 INFO:teuthology.orchestra.run.smithi149.stdout:(14/34): python3-mako-1.1.4-6.el9.noarch.rpm 823 kB/s | 172 kB 00:00 2024-09-10T18:51:28.073 INFO:teuthology.orchestra.run.smithi149.stdout:(15/34): python3-packaging-20.9-5.el9.noarch.rp 577 kB/s | 77 kB 00:00 2024-09-10T18:51:28.099 INFO:teuthology.orchestra.run.smithi149.stdout:(16/34): python3-pytz-2021.1-5.el9.noarch.rpm 676 kB/s | 51 kB 00:00 2024-09-10T18:51:28.106 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-09-10T18:51:28.108 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-10T18:51:28.108 INFO:teuthology.orchestra.run.smithi047.stdout: Package Arch Version Repository Size 2024-09-10T18:51:28.108 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-10T18:51:28.109 INFO:teuthology.orchestra.run.smithi047.stdout:Installing: 2024-09-10T18:51:28.109 INFO:teuthology.orchestra.run.smithi047.stdout: ceph x86_64 2:17.2.7-1606.g5be57395.el9 ceph 6.5 k 2024-09-10T18:51:28.109 INFO:teuthology.orchestra.run.smithi047.stdout:Installing dependencies: 2024-09-10T18:51:28.109 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mds x86_64 2:17.2.7-1606.g5be57395.el9 ceph 2.1 M 2024-09-10T18:51:28.109 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr x86_64 2:17.2.7-1606.g5be57395.el9 ceph 1.4 M 2024-09-10T18:51:28.109 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-modules-core noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 240 k 2024-09-10T18:51:28.109 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mon x86_64 2:17.2.7-1606.g5be57395.el9 ceph 3.9 M 2024-09-10T18:51:28.109 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-osd x86_64 2:17.2.7-1606.g5be57395.el9 ceph 15 M 2024-09-10T18:51:28.109 INFO:teuthology.orchestra.run.smithi047.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-10T18:51:28.109 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-10T18:51:28.109 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-10T18:51:28.109 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-10T18:51:28.110 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-10T18:51:28.110 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-10T18:51:28.110 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-10T18:51:28.110 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-collections 2024-09-10T18:51:28.110 INFO:teuthology.orchestra.run.smithi047.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-09-10T18:51:28.110 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-10T18:51:28.110 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-10T18:51:28.110 INFO:teuthology.orchestra.run.smithi047.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-10T18:51:28.110 INFO:teuthology.orchestra.run.smithi047.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-10T18:51:28.110 INFO:teuthology.orchestra.run.smithi047.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-10T18:51:28.110 INFO:teuthology.orchestra.run.smithi047.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-10T18:51:28.110 INFO:teuthology.orchestra.run.smithi047.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-10T18:51:28.110 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-10T18:51:28.111 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-10T18:51:28.111 INFO:teuthology.orchestra.run.smithi047.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-10T18:51:28.111 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-10T18:51:28.111 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-10T18:51:28.111 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-10T18:51:28.111 INFO:teuthology.orchestra.run.smithi047.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-10T18:51:28.111 INFO:teuthology.orchestra.run.smithi047.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-10T18:51:28.111 INFO:teuthology.orchestra.run.smithi047.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-10T18:51:28.111 INFO:teuthology.orchestra.run.smithi047.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-10T18:51:28.111 INFO:teuthology.orchestra.run.smithi047.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-10T18:51:28.111 INFO:teuthology.orchestra.run.smithi047.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-10T18:51:28.111 INFO:teuthology.orchestra.run.smithi047.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-10T18:51:28.111 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:51:28.111 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction Summary 2024-09-10T18:51:28.112 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-10T18:51:28.112 INFO:teuthology.orchestra.run.smithi047.stdout:Install 34 Packages 2024-09-10T18:51:28.112 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:51:28.112 INFO:teuthology.orchestra.run.smithi047.stdout:Total download size: 27 M 2024-09-10T18:51:28.112 INFO:teuthology.orchestra.run.smithi047.stdout:Installed size: 96 M 2024-09-10T18:51:28.112 INFO:teuthology.orchestra.run.smithi047.stdout:Downloading Packages: 2024-09-10T18:51:28.132 INFO:teuthology.orchestra.run.smithi149.stdout:(17/34): python3-toml-0.10.2-6.el9.noarch.rpm 496 kB/s | 42 kB 00:00 2024-09-10T18:51:28.149 INFO:teuthology.orchestra.run.smithi149.stdout:(18/34): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 577 kB/s | 43 kB 00:00 2024-09-10T18:51:28.183 INFO:teuthology.orchestra.run.smithi149.stdout:(19/34): python3-cheroot-8.6.0-4.el9.noarch.rpm 2.0 MB/s | 172 kB 00:00 2024-09-10T18:51:28.200 INFO:teuthology.orchestra.run.smithi149.stdout:(20/34): python3-jaraco-8.2.1-3.el9.noarch.rpm 251 kB/s | 11 kB 00:00 2024-09-10T18:51:28.225 INFO:teuthology.orchestra.run.smithi149.stdout:(21/34): python3-jaraco-classes-3.2.1-5.el9.noa 764 kB/s | 18 kB 00:00 2024-09-10T18:51:28.242 INFO:teuthology.orchestra.run.smithi149.stdout:(22/34): python3-jaraco-collections-3.0.0-8.el9 580 kB/s | 23 kB 00:00 2024-09-10T18:51:28.259 INFO:teuthology.orchestra.run.smithi149.stdout:(23/34): python3-jaraco-functools-3.5.0-2.el9.n 577 kB/s | 19 kB 00:00 2024-09-10T18:51:28.276 INFO:teuthology.orchestra.run.smithi149.stdout:(24/34): python3-jaraco-text-3.2.0-6.el9.noarch 584 kB/s | 20 kB 00:00 2024-09-10T18:51:28.293 INFO:teuthology.orchestra.run.smithi149.stdout:(25/34): python3-logutils-0.3.5-21.el9.noarch.r 1.4 MB/s | 46 kB 00:00 2024-09-10T18:51:28.318 INFO:teuthology.orchestra.run.smithi149.stdout:(26/34): python3-more-itertools-8.12.0-2.el9.no 1.8 MB/s | 79 kB 00:00 2024-09-10T18:51:28.343 INFO:teuthology.orchestra.run.smithi149.stdout:(27/34): python3-cherrypy-18.6.1-2.el9.noarch.r 1.7 MB/s | 358 kB 00:00 2024-09-10T18:51:28.360 INFO:teuthology.orchestra.run.smithi149.stdout:(28/34): python3-portend-3.1.0-2.el9.noarch.rpm 391 kB/s | 16 kB 00:00 2024-09-10T18:51:28.385 INFO:teuthology.orchestra.run.smithi149.stdout:(29/34): python3-pecan-1.4.2-3.el9.noarch.rpm 2.9 MB/s | 272 kB 00:00 2024-09-10T18:51:28.411 INFO:teuthology.orchestra.run.smithi149.stdout:(30/34): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.3 MB/s | 90 kB 00:00 2024-09-10T18:51:28.428 INFO:teuthology.orchestra.run.smithi149.stdout:(31/34): python3-tempora-5.0.0-2.el9.noarch.rpm 531 kB/s | 36 kB 00:00 2024-09-10T18:51:28.445 INFO:teuthology.orchestra.run.smithi149.stdout:(32/34): python3-zc-lockfile-2.0-10.el9.noarch. 1.2 MB/s | 20 kB 00:00 2024-09-10T18:51:28.478 INFO:teuthology.orchestra.run.smithi149.stdout:(33/34): python3-webob-1.8.8-2.el9.noarch.rpm 2.4 MB/s | 230 kB 00:00 2024-09-10T18:51:28.503 INFO:teuthology.orchestra.run.smithi149.stdout:(34/34): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.5 MB/s | 427 kB 00:00 2024-09-10T18:51:28.508 INFO:teuthology.orchestra.run.smithi149.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:51:28.508 INFO:teuthology.orchestra.run.smithi149.stdout:Total 15 MB/s | 27 MB 00:01 2024-09-10T18:51:28.637 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction check 2024-09-10T18:51:28.673 INFO:teuthology.orchestra.run.smithi047.stdout:(1/34): ceph-17.2.7-1606.g5be57395.el9.x86_64.r 55 kB/s | 6.5 kB 00:00 2024-09-10T18:51:28.675 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction check succeeded. 2024-09-10T18:51:28.675 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction test 2024-09-10T18:51:28.848 INFO:teuthology.orchestra.run.smithi047.stdout:(2/34): ceph-mgr-17.2.7-1606.g5be57395.el9.x86_ 4.8 MB/s | 1.4 MB 00:00 2024-09-10T18:51:28.898 INFO:teuthology.orchestra.run.smithi047.stdout:(3/34): ceph-mds-17.2.7-1606.g5be57395.el9.x86_ 6.0 MB/s | 2.1 MB 00:00 2024-09-10T18:51:28.982 INFO:teuthology.orchestra.run.smithi047.stdout:(4/34): ceph-mgr-modules-core-17.2.7-1606.g5be5 2.8 MB/s | 240 kB 00:00 2024-09-10T18:51:29.055 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction test succeeded. 2024-09-10T18:51:29.055 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction 2024-09-10T18:51:29.065 INFO:teuthology.orchestra.run.smithi047.stdout:(5/34): ceph-mon-17.2.7-1606.g5be57395.el9.x86_ 10 MB/s | 3.9 MB 00:00 2024-09-10T18:51:29.182 INFO:teuthology.orchestra.run.smithi047.stdout:(6/34): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.2 MB/s | 253 kB 00:00 2024-09-10T18:51:29.225 INFO:teuthology.orchestra.run.smithi047.stdout:(7/34): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-09-10T18:51:29.417 INFO:teuthology.orchestra.run.smithi047.stdout:(8/34): ceph-osd-17.2.7-1606.g5be57395.el9.x86_ 27 MB/s | 15 MB 00:00 2024-09-10T18:51:29.442 INFO:teuthology.orchestra.run.smithi047.stdout:(9/34): python3-pycparser-2.20-6.el9.noarch.rpm 623 kB/s | 135 kB 00:00 2024-09-10T18:51:29.475 INFO:teuthology.orchestra.run.smithi047.stdout:(10/34): python3-cryptography-36.0.1-4.el9.x86_ 3.0 MB/s | 1.2 MB 00:00 2024-09-10T18:51:29.500 INFO:teuthology.orchestra.run.smithi047.stdout:(11/34): python3-urllib3-1.26.5-6.el9.noarch.rp 3.6 MB/s | 215 kB 00:00 2024-09-10T18:51:29.551 INFO:teuthology.orchestra.run.smithi047.stdout:(12/34): python3-requests-2.25.1-8.el9.noarch.r 931 kB/s | 125 kB 00:00 2024-09-10T18:51:29.793 INFO:teuthology.orchestra.run.smithi047.stdout:(13/34): python3-markupsafe-1.1.1-12.el9.x86_64 119 kB/s | 35 kB 00:00 2024-09-10T18:51:29.927 INFO:teuthology.orchestra.run.smithi047.stdout:(14/34): python3-packaging-20.9-5.el9.noarch.rp 205 kB/s | 77 kB 00:00 2024-09-10T18:51:29.994 INFO:teuthology.orchestra.run.smithi047.stdout:(15/34): python3-pytz-2021.1-5.el9.noarch.rpm 254 kB/s | 51 kB 00:00 2024-09-10T18:51:30.022 INFO:teuthology.orchestra.run.smithi047.stdout:(16/34): python3-mako-1.1.4-6.el9.noarch.rpm 315 kB/s | 172 kB 00:00 2024-09-10T18:51:30.056 INFO:teuthology.orchestra.run.smithi047.stdout:(17/34): python3-toml-0.10.2-6.el9.noarch.rpm 323 kB/s | 42 kB 00:00 2024-09-10T18:51:30.089 INFO:teuthology.orchestra.run.smithi047.stdout:(18/34): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 456 kB/s | 43 kB 00:00 2024-09-10T18:51:30.123 INFO:teuthology.orchestra.run.smithi047.stdout:(19/34): python3-jaraco-8.2.1-3.el9.noarch.rpm 449 kB/s | 11 kB 00:00 2024-09-10T18:51:30.148 INFO:teuthology.orchestra.run.smithi047.stdout:(20/34): python3-jaraco-classes-3.2.1-5.el9.noa 768 kB/s | 18 kB 00:00 2024-09-10T18:51:30.173 INFO:teuthology.orchestra.run.smithi047.stdout:(21/34): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.2 MB/s | 172 kB 00:00 2024-09-10T18:51:30.199 INFO:teuthology.orchestra.run.smithi047.stdout:(22/34): python3-jaraco-collections-3.0.0-8.el9 460 kB/s | 23 kB 00:00 2024-09-10T18:51:30.224 INFO:teuthology.orchestra.run.smithi047.stdout:(23/34): python3-jaraco-functools-3.5.0-2.el9.n 386 kB/s | 19 kB 00:00 2024-09-10T18:51:30.249 INFO:teuthology.orchestra.run.smithi047.stdout:(24/34): python3-jaraco-text-3.2.0-6.el9.noarch 392 kB/s | 20 kB 00:00 2024-09-10T18:51:30.275 INFO:teuthology.orchestra.run.smithi047.stdout:(25/34): python3-logutils-0.3.5-21.el9.noarch.r 919 kB/s | 46 kB 00:00 2024-09-10T18:51:30.300 INFO:teuthology.orchestra.run.smithi047.stdout:(26/34): python3-more-itertools-8.12.0-2.el9.no 1.5 MB/s | 79 kB 00:00 2024-09-10T18:51:30.302 INFO:teuthology.orchestra.run.smithi149.stdout: Preparing : 1/1 2024-09-10T18:51:30.325 INFO:teuthology.orchestra.run.smithi047.stdout:(27/34): python3-portend-3.1.0-2.el9.noarch.rpm 654 kB/s | 16 kB 00:00 2024-09-10T18:51:30.365 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/34 2024-09-10T18:51:30.380 INFO:teuthology.orchestra.run.smithi047.stdout:(28/34): python3-cherrypy-18.6.1-2.el9.noarch.r 1.1 MB/s | 358 kB 00:00 2024-09-10T18:51:30.397 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/34 2024-09-10T18:51:30.409 INFO:teuthology.orchestra.run.smithi047.stdout:(29/34): python3-tempora-5.0.0-2.el9.noarch.rpm 1.2 MB/s | 36 kB 00:00 2024-09-10T18:51:30.430 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/34 2024-09-10T18:51:30.434 INFO:teuthology.orchestra.run.smithi047.stdout:(30/34): python3-pyOpenSSL-21.0.0-1.el9.noarch. 826 kB/s | 90 kB 00:00 2024-09-10T18:51:30.460 INFO:teuthology.orchestra.run.smithi047.stdout:(31/34): python3-pecan-1.4.2-3.el9.noarch.rpm 1.4 MB/s | 272 kB 00:00 2024-09-10T18:51:30.488 INFO:teuthology.orchestra.run.smithi047.stdout:(32/34): python3-zc-lockfile-2.0-10.el9.noarch. 804 kB/s | 20 kB 00:00 2024-09-10T18:51:30.510 INFO:teuthology.orchestra.run.smithi047.stdout:(33/34): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-10T18:51:30.522 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/34 2024-09-10T18:51:30.544 INFO:teuthology.orchestra.run.smithi047.stdout:(34/34): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.8 MB/s | 427 kB 00:00 2024-09-10T18:51:30.548 INFO:teuthology.orchestra.run.smithi047.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:51:30.548 INFO:teuthology.orchestra.run.smithi047.stdout:Total 11 MB/s | 27 MB 00:02 2024-09-10T18:51:30.609 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/34 2024-09-10T18:51:30.650 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 6/34 2024-09-10T18:51:30.681 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction check 2024-09-10T18:51:30.685 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/34 2024-09-10T18:51:30.715 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/34 2024-09-10T18:51:30.721 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction check succeeded. 2024-09-10T18:51:30.721 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction test 2024-09-10T18:51:30.743 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/34 2024-09-10T18:51:30.777 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/34 2024-09-10T18:51:30.817 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/34 2024-09-10T18:51:30.861 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/34 2024-09-10T18:51:30.904 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/34 2024-09-10T18:51:30.952 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/34 2024-09-10T18:51:30.988 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-packaging-20.9-5.el9.noarch 15/34 2024-09-10T18:51:31.066 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 16/34 2024-09-10T18:51:31.113 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction test succeeded. 2024-09-10T18:51:31.113 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction 2024-09-10T18:51:31.214 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 17/34 2024-09-10T18:51:31.292 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 18/34 2024-09-10T18:51:31.349 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 19/34 2024-09-10T18:51:31.393 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 20/34 2024-09-10T18:51:31.450 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-ply-3.11-14.el9.noarch 21/34 2024-09-10T18:51:31.505 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 22/34 2024-09-10T18:51:31.701 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 23/34 2024-09-10T18:51:31.746 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 24/34 2024-09-10T18:51:31.825 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 25/34 2024-09-10T18:51:31.922 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 26/34 2024-09-10T18:51:31.965 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 27/34 2024-09-10T18:51:32.087 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 28/34 2024-09-10T18:51:32.242 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-1606.g5be57395.el9. 29/34 2024-09-10T18:51:32.273 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-2:17.2.7-1606.g5be57395.el9.x86_64 30/34 2024-09-10T18:51:32.298 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mgr-2:17.2.7-1606.g5be57395.el9.x86_64 30/34 2024-09-10T18:51:32.298 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T18:51:32.299 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-10T18:51:32.299 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T18:51:32.299 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T18:51:32.299 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T18:51:32.557 INFO:teuthology.orchestra.run.smithi047.stdout: Preparing : 1/1 2024-09-10T18:51:32.630 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/34 2024-09-10T18:51:32.672 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/34 2024-09-10T18:51:32.713 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/34 2024-09-10T18:51:32.808 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/34 2024-09-10T18:51:32.885 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/34 2024-09-10T18:51:32.933 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 6/34 2024-09-10T18:51:32.976 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/34 2024-09-10T18:51:33.005 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/34 2024-09-10T18:51:33.040 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/34 2024-09-10T18:51:33.073 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/34 2024-09-10T18:51:33.114 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/34 2024-09-10T18:51:33.157 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/34 2024-09-10T18:51:33.191 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/34 2024-09-10T18:51:33.241 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/34 2024-09-10T18:51:33.296 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-packaging-20.9-5.el9.noarch 15/34 2024-09-10T18:51:33.382 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 16/34 2024-09-10T18:51:33.533 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 17/34 2024-09-10T18:51:33.626 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 18/34 2024-09-10T18:51:33.674 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-osd-2:17.2.7-1606.g5be57395.el9.x86_64 31/34 2024-09-10T18:51:33.684 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 19/34 2024-09-10T18:51:33.704 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-osd-2:17.2.7-1606.g5be57395.el9.x86_64 31/34 2024-09-10T18:51:33.704 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T18:51:33.705 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-10T18:51:33.705 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T18:51:33.705 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T18:51:33.705 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T18:51:33.722 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 20/34 2024-09-10T18:51:33.780 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-ply-3.11-14.el9.noarch 21/34 2024-09-10T18:51:33.836 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 22/34 2024-09-10T18:51:34.031 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 23/34 2024-09-10T18:51:34.084 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mon-2:17.2.7-1606.g5be57395.el9.x86_64 32/34 2024-09-10T18:51:34.093 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 24/34 2024-09-10T18:51:34.108 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mon-2:17.2.7-1606.g5be57395.el9.x86_64 32/34 2024-09-10T18:51:34.108 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T18:51:34.108 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-10T18:51:34.108 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T18:51:34.108 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T18:51:34.109 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T18:51:34.176 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 25/34 2024-09-10T18:51:34.296 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 26/34 2024-09-10T18:51:34.309 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mds-2:17.2.7-1606.g5be57395.el9.x86_64 33/34 2024-09-10T18:51:34.333 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mds-2:17.2.7-1606.g5be57395.el9.x86_64 33/34 2024-09-10T18:51:34.334 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T18:51:34.334 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-10T18:51:34.334 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T18:51:34.334 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T18:51:34.334 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T18:51:34.339 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 27/34 2024-09-10T18:51:34.398 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-2:17.2.7-1606.g5be57395.el9.x86_64 34/34 2024-09-10T18:51:34.474 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 28/34 2024-09-10T18:51:34.636 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-1606.g5be57395.el9. 29/34 2024-09-10T18:51:34.668 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mgr-2:17.2.7-1606.g5be57395.el9.x86_64 30/34 2024-09-10T18:51:34.696 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mgr-2:17.2.7-1606.g5be57395.el9.x86_64 30/34 2024-09-10T18:51:34.696 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T18:51:34.696 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-10T18:51:34.696 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T18:51:34.697 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T18:51:34.697 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:51:35.995 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-2:17.2.7-1606.g5be57395.el9.x86_64 34/34 2024-09-10T18:51:35.995 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-2:17.2.7-1606.g5be57395.el9.x86_64 1/34 2024-09-10T18:51:35.995 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mds-2:17.2.7-1606.g5be57395.el9.x86_64 2/34 2024-09-10T18:51:35.995 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-2:17.2.7-1606.g5be57395.el9.x86_64 3/34 2024-09-10T18:51:35.995 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mon-2:17.2.7-1606.g5be57395.el9.x86_64 4/34 2024-09-10T18:51:35.995 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-osd-2:17.2.7-1606.g5be57395.el9.x86_64 5/34 2024-09-10T18:51:35.995 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-1606.g5be57395.el9. 6/34 2024-09-10T18:51:35.996 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/34 2024-09-10T18:51:35.996 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/34 2024-09-10T18:51:35.996 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/34 2024-09-10T18:51:35.996 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/34 2024-09-10T18:51:35.996 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/34 2024-09-10T18:51:35.996 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/34 2024-09-10T18:51:35.996 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/34 2024-09-10T18:51:35.996 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/34 2024-09-10T18:51:35.996 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 15/34 2024-09-10T18:51:35.996 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 16/34 2024-09-10T18:51:35.996 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 17/34 2024-09-10T18:51:35.996 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 18/34 2024-09-10T18:51:35.996 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 19/34 2024-09-10T18:51:35.997 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 20/34 2024-09-10T18:51:35.997 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 21/34 2024-09-10T18:51:35.997 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 22/34 2024-09-10T18:51:35.997 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 23/34 2024-09-10T18:51:35.997 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 24/34 2024-09-10T18:51:35.997 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 25/34 2024-09-10T18:51:35.997 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 26/34 2024-09-10T18:51:35.997 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 27/34 2024-09-10T18:51:35.997 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 28/34 2024-09-10T18:51:35.998 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 29/34 2024-09-10T18:51:35.998 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 30/34 2024-09-10T18:51:35.998 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 31/34 2024-09-10T18:51:35.998 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 32/34 2024-09-10T18:51:35.998 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 33/34 2024-09-10T18:51:36.086 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-osd-2:17.2.7-1606.g5be57395.el9.x86_64 31/34 2024-09-10T18:51:36.115 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-osd-2:17.2.7-1606.g5be57395.el9.x86_64 31/34 2024-09-10T18:51:36.115 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T18:51:36.115 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-10T18:51:36.115 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T18:51:36.116 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T18:51:36.116 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:51:36.520 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mon-2:17.2.7-1606.g5be57395.el9.x86_64 32/34 2024-09-10T18:51:36.545 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mon-2:17.2.7-1606.g5be57395.el9.x86_64 32/34 2024-09-10T18:51:36.545 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T18:51:36.545 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-10T18:51:36.545 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T18:51:36.545 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T18:51:36.545 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:51:36.762 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mds-2:17.2.7-1606.g5be57395.el9.x86_64 33/34 2024-09-10T18:51:36.787 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mds-2:17.2.7-1606.g5be57395.el9.x86_64 33/34 2024-09-10T18:51:36.788 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T18:51:36.788 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-10T18:51:36.788 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T18:51:36.788 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T18:51:36.788 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:51:36.854 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-2:17.2.7-1606.g5be57395.el9.x86_64 34/34 2024-09-10T18:51:36.893 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 34/34 2024-09-10T18:51:36.893 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T18:51:36.893 INFO:teuthology.orchestra.run.smithi149.stdout:Installed: 2024-09-10T18:51:36.893 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:51:36.893 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mds-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:51:36.893 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:51:36.893 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-modules-core-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:51:36.893 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mon-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:51:36.894 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-osd-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:51:36.894 INFO:teuthology.orchestra.run.smithi149.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-10T18:51:36.894 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-10T18:51:36.894 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-10T18:51:36.894 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-10T18:51:36.894 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-10T18:51:36.894 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-10T18:51:36.894 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-10T18:51:36.894 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-10T18:51:36.894 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-10T18:51:36.894 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-10T18:51:36.895 INFO:teuthology.orchestra.run.smithi149.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-10T18:51:36.895 INFO:teuthology.orchestra.run.smithi149.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-10T18:51:36.895 INFO:teuthology.orchestra.run.smithi149.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-10T18:51:36.895 INFO:teuthology.orchestra.run.smithi149.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-10T18:51:36.895 INFO:teuthology.orchestra.run.smithi149.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-10T18:51:36.895 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-10T18:51:36.895 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ply-3.11-14.el9.noarch 2024-09-10T18:51:36.895 INFO:teuthology.orchestra.run.smithi149.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-10T18:51:36.895 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-10T18:51:36.895 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-10T18:51:36.895 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-10T18:51:36.896 INFO:teuthology.orchestra.run.smithi149.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-10T18:51:36.896 INFO:teuthology.orchestra.run.smithi149.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-10T18:51:36.896 INFO:teuthology.orchestra.run.smithi149.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-10T18:51:36.896 INFO:teuthology.orchestra.run.smithi149.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-10T18:51:36.896 INFO:teuthology.orchestra.run.smithi149.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-10T18:51:36.896 INFO:teuthology.orchestra.run.smithi149.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-10T18:51:36.896 INFO:teuthology.orchestra.run.smithi149.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-10T18:51:36.896 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T18:51:36.896 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-09-10T18:51:37.270 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install ceph-base 2024-09-10T18:51:37.851 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:00:49 ago on Tue 10 Sep 2024 06:50:48 PM UTC. 2024-09-10T18:51:37.943 INFO:teuthology.orchestra.run.smithi149.stdout:Package ceph-base-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T18:51:37.985 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-09-10T18:51:37.986 INFO:teuthology.orchestra.run.smithi149.stdout:Nothing to do. 2024-09-10T18:51:37.986 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-09-10T18:51:38.059 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install cephadm 2024-09-10T18:51:38.421 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-2:17.2.7-1606.g5be57395.el9.x86_64 34/34 2024-09-10T18:51:38.421 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-2:17.2.7-1606.g5be57395.el9.x86_64 1/34 2024-09-10T18:51:38.421 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mds-2:17.2.7-1606.g5be57395.el9.x86_64 2/34 2024-09-10T18:51:38.421 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-2:17.2.7-1606.g5be57395.el9.x86_64 3/34 2024-09-10T18:51:38.421 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mon-2:17.2.7-1606.g5be57395.el9.x86_64 4/34 2024-09-10T18:51:38.421 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-osd-2:17.2.7-1606.g5be57395.el9.x86_64 5/34 2024-09-10T18:51:38.421 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-1606.g5be57395.el9. 6/34 2024-09-10T18:51:38.421 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/34 2024-09-10T18:51:38.421 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/34 2024-09-10T18:51:38.421 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/34 2024-09-10T18:51:38.421 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/34 2024-09-10T18:51:38.422 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/34 2024-09-10T18:51:38.422 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/34 2024-09-10T18:51:38.422 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/34 2024-09-10T18:51:38.422 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/34 2024-09-10T18:51:38.422 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 15/34 2024-09-10T18:51:38.422 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 16/34 2024-09-10T18:51:38.422 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 17/34 2024-09-10T18:51:38.422 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 18/34 2024-09-10T18:51:38.422 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 19/34 2024-09-10T18:51:38.422 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 20/34 2024-09-10T18:51:38.422 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 21/34 2024-09-10T18:51:38.422 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 22/34 2024-09-10T18:51:38.423 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 23/34 2024-09-10T18:51:38.423 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 24/34 2024-09-10T18:51:38.423 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 25/34 2024-09-10T18:51:38.423 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 26/34 2024-09-10T18:51:38.423 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 27/34 2024-09-10T18:51:38.423 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 28/34 2024-09-10T18:51:38.423 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 29/34 2024-09-10T18:51:38.423 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 30/34 2024-09-10T18:51:38.423 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 31/34 2024-09-10T18:51:38.423 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 32/34 2024-09-10T18:51:38.423 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 33/34 2024-09-10T18:51:38.629 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:00:50 ago on Tue 10 Sep 2024 06:50:48 PM UTC. 2024-09-10T18:51:38.762 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-09-10T18:51:38.762 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-09-10T18:51:38.763 INFO:teuthology.orchestra.run.smithi149.stdout: Package Arch Version Repository Size 2024-09-10T18:51:38.763 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-09-10T18:51:38.763 INFO:teuthology.orchestra.run.smithi149.stdout:Installing: 2024-09-10T18:51:38.763 INFO:teuthology.orchestra.run.smithi149.stdout: cephadm noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 83 k 2024-09-10T18:51:38.763 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T18:51:38.763 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction Summary 2024-09-10T18:51:38.763 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-09-10T18:51:38.763 INFO:teuthology.orchestra.run.smithi149.stdout:Install 1 Package 2024-09-10T18:51:38.763 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T18:51:38.763 INFO:teuthology.orchestra.run.smithi149.stdout:Total download size: 83 k 2024-09-10T18:51:38.764 INFO:teuthology.orchestra.run.smithi149.stdout:Installed size: 375 k 2024-09-10T18:51:38.764 INFO:teuthology.orchestra.run.smithi149.stdout:Downloading Packages: 2024-09-10T18:51:38.952 INFO:teuthology.orchestra.run.smithi149.stdout:cephadm-17.2.7-1606.g5be57395.el9.noarch.rpm 440 kB/s | 83 kB 00:00 2024-09-10T18:51:38.952 INFO:teuthology.orchestra.run.smithi149.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:51:38.952 INFO:teuthology.orchestra.run.smithi149.stdout:Total 436 kB/s | 83 kB 00:00 2024-09-10T18:51:38.953 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction check 2024-09-10T18:51:38.958 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction check succeeded. 2024-09-10T18:51:38.958 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction test 2024-09-10T18:51:38.960 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction test succeeded. 2024-09-10T18:51:38.960 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction 2024-09-10T18:51:39.040 INFO:teuthology.orchestra.run.smithi149.stdout: Preparing : 1/1 2024-09-10T18:51:39.332 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 34/34 2024-09-10T18:51:39.332 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:51:39.332 INFO:teuthology.orchestra.run.smithi047.stdout:Installed: 2024-09-10T18:51:39.333 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:51:39.333 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mds-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:51:39.333 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:51:39.333 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-modules-core-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:51:39.333 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mon-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:51:39.333 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-osd-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:51:39.333 INFO:teuthology.orchestra.run.smithi047.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-10T18:51:39.333 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-10T18:51:39.333 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-10T18:51:39.333 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-10T18:51:39.333 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-10T18:51:39.333 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-10T18:51:39.333 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-10T18:51:39.334 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-10T18:51:39.334 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-10T18:51:39.334 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-10T18:51:39.334 INFO:teuthology.orchestra.run.smithi047.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-10T18:51:39.334 INFO:teuthology.orchestra.run.smithi047.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-10T18:51:39.334 INFO:teuthology.orchestra.run.smithi047.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-10T18:51:39.334 INFO:teuthology.orchestra.run.smithi047.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-10T18:51:39.334 INFO:teuthology.orchestra.run.smithi047.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-10T18:51:39.334 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-10T18:51:39.334 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ply-3.11-14.el9.noarch 2024-09-10T18:51:39.334 INFO:teuthology.orchestra.run.smithi047.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-10T18:51:39.334 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-10T18:51:39.334 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-10T18:51:39.335 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-10T18:51:39.335 INFO:teuthology.orchestra.run.smithi047.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-10T18:51:39.335 INFO:teuthology.orchestra.run.smithi047.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-10T18:51:39.335 INFO:teuthology.orchestra.run.smithi047.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-10T18:51:39.335 INFO:teuthology.orchestra.run.smithi047.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-10T18:51:39.335 INFO:teuthology.orchestra.run.smithi047.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-10T18:51:39.335 INFO:teuthology.orchestra.run.smithi047.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-10T18:51:39.335 INFO:teuthology.orchestra.run.smithi047.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-10T18:51:39.335 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:51:39.335 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-09-10T18:51:39.459 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: cephadm-2:17.2.7-1606.g5be57395.el9.noarch 1/1 2024-09-10T18:51:39.538 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : cephadm-2:17.2.7-1606.g5be57395.el9.noarch 1/1 2024-09-10T18:51:39.748 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install ceph-base 2024-09-10T18:51:39.929 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: cephadm-2:17.2.7-1606.g5be57395.el9.noarch 1/1 2024-09-10T18:51:40.181 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : cephadm-2:17.2.7-1606.g5be57395.el9.noarch 1/1 2024-09-10T18:51:40.181 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T18:51:40.181 INFO:teuthology.orchestra.run.smithi149.stdout:Installed: 2024-09-10T18:51:40.181 INFO:teuthology.orchestra.run.smithi149.stdout: cephadm-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:51:40.181 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T18:51:40.182 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-09-10T18:51:40.341 INFO:teuthology.orchestra.run.smithi047.stdout:Last metadata expiration check: 0:00:53 ago on Tue 10 Sep 2024 06:50:47 PM UTC. 2024-09-10T18:51:40.422 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install ceph-immutable-object-cache 2024-09-10T18:51:40.434 INFO:teuthology.orchestra.run.smithi047.stdout:Package ceph-base-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T18:51:40.478 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-09-10T18:51:40.479 INFO:teuthology.orchestra.run.smithi047.stdout:Nothing to do. 2024-09-10T18:51:40.479 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-09-10T18:51:40.544 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install cephadm 2024-09-10T18:51:41.008 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:00:53 ago on Tue 10 Sep 2024 06:50:48 PM UTC. 2024-09-10T18:51:41.133 INFO:teuthology.orchestra.run.smithi047.stdout:Last metadata expiration check: 0:00:54 ago on Tue 10 Sep 2024 06:50:47 PM UTC. 2024-09-10T18:51:41.144 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-09-10T18:51:41.144 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-09-10T18:51:41.145 INFO:teuthology.orchestra.run.smithi149.stdout: Package Arch Version Repo Size 2024-09-10T18:51:41.145 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-09-10T18:51:41.145 INFO:teuthology.orchestra.run.smithi149.stdout:Installing: 2024-09-10T18:51:41.145 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-1606.g5be57395.el9 ceph 137 k 2024-09-10T18:51:41.145 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T18:51:41.145 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction Summary 2024-09-10T18:51:41.145 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-09-10T18:51:41.145 INFO:teuthology.orchestra.run.smithi149.stdout:Install 1 Package 2024-09-10T18:51:41.145 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T18:51:41.146 INFO:teuthology.orchestra.run.smithi149.stdout:Total download size: 137 k 2024-09-10T18:51:41.146 INFO:teuthology.orchestra.run.smithi149.stdout:Installed size: 379 k 2024-09-10T18:51:41.146 INFO:teuthology.orchestra.run.smithi149.stdout:Downloading Packages: 2024-09-10T18:51:41.270 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-09-10T18:51:41.270 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-10T18:51:41.271 INFO:teuthology.orchestra.run.smithi047.stdout: Package Arch Version Repository Size 2024-09-10T18:51:41.271 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-10T18:51:41.271 INFO:teuthology.orchestra.run.smithi047.stdout:Installing: 2024-09-10T18:51:41.271 INFO:teuthology.orchestra.run.smithi047.stdout: cephadm noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 83 k 2024-09-10T18:51:41.271 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:51:41.271 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction Summary 2024-09-10T18:51:41.271 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-10T18:51:41.271 INFO:teuthology.orchestra.run.smithi047.stdout:Install 1 Package 2024-09-10T18:51:41.271 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:51:41.272 INFO:teuthology.orchestra.run.smithi047.stdout:Total download size: 83 k 2024-09-10T18:51:41.272 INFO:teuthology.orchestra.run.smithi047.stdout:Installed size: 375 k 2024-09-10T18:51:41.272 INFO:teuthology.orchestra.run.smithi047.stdout:Downloading Packages: 2024-09-10T18:51:41.358 INFO:teuthology.orchestra.run.smithi149.stdout:ceph-immutable-object-cache-17.2.7-1606.g5be573 649 kB/s | 137 kB 00:00 2024-09-10T18:51:41.359 INFO:teuthology.orchestra.run.smithi149.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:51:41.359 INFO:teuthology.orchestra.run.smithi149.stdout:Total 643 kB/s | 137 kB 00:00 2024-09-10T18:51:41.359 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction check 2024-09-10T18:51:41.367 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction check succeeded. 2024-09-10T18:51:41.367 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction test 2024-09-10T18:51:41.429 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction test succeeded. 2024-09-10T18:51:41.429 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction 2024-09-10T18:51:41.461 INFO:teuthology.orchestra.run.smithi047.stdout:cephadm-17.2.7-1606.g5be57395.el9.noarch.rpm 435 kB/s | 83 kB 00:00 2024-09-10T18:51:41.462 INFO:teuthology.orchestra.run.smithi047.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:51:41.462 INFO:teuthology.orchestra.run.smithi047.stdout:Total 431 kB/s | 83 kB 00:00 2024-09-10T18:51:41.462 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction check 2024-09-10T18:51:41.468 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction check succeeded. 2024-09-10T18:51:41.468 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction test 2024-09-10T18:51:41.470 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction test succeeded. 2024-09-10T18:51:41.470 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction 2024-09-10T18:51:41.546 INFO:teuthology.orchestra.run.smithi047.stdout: Preparing : 1/1 2024-09-10T18:51:41.582 INFO:teuthology.orchestra.run.smithi149.stdout: Preparing : 1/1 2024-09-10T18:51:41.651 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-1606.g5be57395. 1/1 2024-09-10T18:51:41.676 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-1606.g5be57395. 1/1 2024-09-10T18:51:41.676 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T18:51:41.676 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-10T18:51:41.676 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T18:51:41.981 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: cephadm-2:17.2.7-1606.g5be57395.el9.noarch 1/1 2024-09-10T18:51:42.069 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : cephadm-2:17.2.7-1606.g5be57395.el9.noarch 1/1 2024-09-10T18:51:42.420 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: cephadm-2:17.2.7-1606.g5be57395.el9.noarch 1/1 2024-09-10T18:51:42.592 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-1606.g5be57395. 1/1 2024-09-10T18:51:42.593 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T18:51:42.593 INFO:teuthology.orchestra.run.smithi149.stdout:Installed: 2024-09-10T18:51:42.593 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-immutable-object-cache-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:51:42.593 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T18:51:42.593 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-09-10T18:51:42.674 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : cephadm-2:17.2.7-1606.g5be57395.el9.noarch 1/1 2024-09-10T18:51:42.675 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:51:42.675 INFO:teuthology.orchestra.run.smithi047.stdout:Installed: 2024-09-10T18:51:42.675 INFO:teuthology.orchestra.run.smithi047.stdout: cephadm-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:51:42.675 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:51:42.675 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-09-10T18:51:42.835 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install ceph-mgr 2024-09-10T18:51:42.875 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install ceph-immutable-object-cache 2024-09-10T18:51:43.429 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:00:55 ago on Tue 10 Sep 2024 06:50:48 PM UTC. 2024-09-10T18:51:43.470 INFO:teuthology.orchestra.run.smithi047.stdout:Last metadata expiration check: 0:00:56 ago on Tue 10 Sep 2024 06:50:47 PM UTC. 2024-09-10T18:51:43.519 INFO:teuthology.orchestra.run.smithi149.stdout:Package ceph-mgr-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T18:51:43.560 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-09-10T18:51:43.561 INFO:teuthology.orchestra.run.smithi149.stdout:Nothing to do. 2024-09-10T18:51:43.562 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-09-10T18:51:43.610 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-09-10T18:51:43.611 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-10T18:51:43.611 INFO:teuthology.orchestra.run.smithi047.stdout: Package Arch Version Repo Size 2024-09-10T18:51:43.611 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-10T18:51:43.612 INFO:teuthology.orchestra.run.smithi047.stdout:Installing: 2024-09-10T18:51:43.612 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-1606.g5be57395.el9 ceph 137 k 2024-09-10T18:51:43.612 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:51:43.612 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction Summary 2024-09-10T18:51:43.612 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-10T18:51:43.612 INFO:teuthology.orchestra.run.smithi047.stdout:Install 1 Package 2024-09-10T18:51:43.612 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:51:43.612 INFO:teuthology.orchestra.run.smithi047.stdout:Total download size: 137 k 2024-09-10T18:51:43.612 INFO:teuthology.orchestra.run.smithi047.stdout:Installed size: 379 k 2024-09-10T18:51:43.612 INFO:teuthology.orchestra.run.smithi047.stdout:Downloading Packages: 2024-09-10T18:51:43.630 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install ceph-mgr-dashboard 2024-09-10T18:51:43.827 INFO:teuthology.orchestra.run.smithi047.stdout:ceph-immutable-object-cache-17.2.7-1606.g5be573 643 kB/s | 137 kB 00:00 2024-09-10T18:51:43.827 INFO:teuthology.orchestra.run.smithi047.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:51:43.827 INFO:teuthology.orchestra.run.smithi047.stdout:Total 638 kB/s | 137 kB 00:00 2024-09-10T18:51:43.827 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction check 2024-09-10T18:51:43.835 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction check succeeded. 2024-09-10T18:51:43.835 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction test 2024-09-10T18:51:43.901 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction test succeeded. 2024-09-10T18:51:43.902 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction 2024-09-10T18:51:44.056 INFO:teuthology.orchestra.run.smithi047.stdout: Preparing : 1/1 2024-09-10T18:51:44.127 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-1606.g5be57395. 1/1 2024-09-10T18:51:44.154 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-1606.g5be57395. 1/1 2024-09-10T18:51:44.154 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T18:51:44.154 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-10T18:51:44.154 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:51:44.211 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:00:56 ago on Tue 10 Sep 2024 06:50:48 PM UTC. 2024-09-10T18:51:44.345 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-09-10T18:51:44.345 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-09-10T18:51:44.345 INFO:teuthology.orchestra.run.smithi149.stdout: Package Arch Version Repository Size 2024-09-10T18:51:44.346 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-09-10T18:51:44.346 INFO:teuthology.orchestra.run.smithi149.stdout:Installing: 2024-09-10T18:51:44.346 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-dashboard noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 3.4 M 2024-09-10T18:51:44.346 INFO:teuthology.orchestra.run.smithi149.stdout:Installing dependencies: 2024-09-10T18:51:44.346 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-grafana-dashboards noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 24 k 2024-09-10T18:51:44.346 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-prometheus-alerts noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 15 k 2024-09-10T18:51:44.346 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-09-10T18:51:44.346 INFO:teuthology.orchestra.run.smithi149.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-10T18:51:44.346 INFO:teuthology.orchestra.run.smithi149.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-10T18:51:44.346 INFO:teuthology.orchestra.run.smithi149.stdout:Installing weak dependencies: 2024-09-10T18:51:44.346 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-09-10T18:51:44.346 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T18:51:44.347 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction Summary 2024-09-10T18:51:44.347 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-09-10T18:51:44.347 INFO:teuthology.orchestra.run.smithi149.stdout:Install 7 Packages 2024-09-10T18:51:44.347 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T18:51:44.347 INFO:teuthology.orchestra.run.smithi149.stdout:Total download size: 3.7 M 2024-09-10T18:51:44.347 INFO:teuthology.orchestra.run.smithi149.stdout:Installed size: 80 M 2024-09-10T18:51:44.348 INFO:teuthology.orchestra.run.smithi149.stdout:Downloading Packages: 2024-09-10T18:51:44.679 INFO:teuthology.orchestra.run.smithi149.stdout:(1/7): ceph-grafana-dashboards-17.2.7-1606.g5be 163 kB/s | 24 kB 00:00 2024-09-10T18:51:44.704 INFO:teuthology.orchestra.run.smithi149.stdout:(2/7): ceph-prometheus-alerts-17.2.7-1606.g5be5 83 kB/s | 15 kB 00:00 2024-09-10T18:51:44.746 INFO:teuthology.orchestra.run.smithi149.stdout:(3/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 135 kB/s | 9.0 kB 00:00 2024-09-10T18:51:44.771 INFO:teuthology.orchestra.run.smithi149.stdout:(4/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 1.3 MB/s | 31 kB 00:00 2024-09-10T18:51:44.797 INFO:teuthology.orchestra.run.smithi149.stdout:(5/7): python3-jwt-2.4.0-1.el9.noarch.rpm 443 kB/s | 41 kB 00:00 2024-09-10T18:51:44.822 INFO:teuthology.orchestra.run.smithi149.stdout:(6/7): python3-routes-2.5.1-5.el9.noarch.rpm 3.7 MB/s | 188 kB 00:00 2024-09-10T18:51:44.922 INFO:teuthology.orchestra.run.smithi149.stdout:(7/7): ceph-mgr-dashboard-17.2.7-1606.g5be57395 8.5 MB/s | 3.4 MB 00:00 2024-09-10T18:51:44.926 INFO:teuthology.orchestra.run.smithi149.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:51:44.926 INFO:teuthology.orchestra.run.smithi149.stdout:Total 6.3 MB/s | 3.7 MB 00:00 2024-09-10T18:51:44.926 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction check 2024-09-10T18:51:44.934 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction check succeeded. 2024-09-10T18:51:44.935 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction test 2024-09-10T18:51:45.024 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction test succeeded. 2024-09-10T18:51:45.025 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction 2024-09-10T18:51:45.127 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-1606.g5be57395. 1/1 2024-09-10T18:51:45.127 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:51:45.127 INFO:teuthology.orchestra.run.smithi047.stdout:Installed: 2024-09-10T18:51:45.127 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-immutable-object-cache-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:51:45.127 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:51:45.127 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-09-10T18:51:45.314 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install ceph-mgr 2024-09-10T18:51:45.337 INFO:teuthology.orchestra.run.smithi149.stdout: Preparing : 1/1 2024-09-10T18:51:45.415 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-09-10T18:51:45.449 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-09-10T18:51:45.495 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-09-10T18:51:45.528 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-09-10T18:51:45.557 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-1606.g5be57395.el9.n 5/7 2024-09-10T18:51:45.910 INFO:teuthology.orchestra.run.smithi047.stdout:Last metadata expiration check: 0:00:58 ago on Tue 10 Sep 2024 06:50:47 PM UTC. 2024-09-10T18:51:46.006 INFO:teuthology.orchestra.run.smithi047.stdout:Package ceph-mgr-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T18:51:46.050 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-09-10T18:51:46.051 INFO:teuthology.orchestra.run.smithi047.stdout:Nothing to do. 2024-09-10T18:51:46.051 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-09-10T18:51:46.119 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install ceph-mgr-dashboard 2024-09-10T18:51:46.566 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-1606.g5be57395.el9. 6/7 2024-09-10T18:51:46.611 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-1606.g5be57395.el9.noarc 7/7 2024-09-10T18:51:46.701 INFO:teuthology.orchestra.run.smithi047.stdout:Last metadata expiration check: 0:00:59 ago on Tue 10 Sep 2024 06:50:47 PM UTC. 2024-09-10T18:51:46.839 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-09-10T18:51:46.840 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-10T18:51:46.840 INFO:teuthology.orchestra.run.smithi047.stdout: Package Arch Version Repository Size 2024-09-10T18:51:46.840 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-10T18:51:46.840 INFO:teuthology.orchestra.run.smithi047.stdout:Installing: 2024-09-10T18:51:46.841 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-dashboard noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 3.4 M 2024-09-10T18:51:46.841 INFO:teuthology.orchestra.run.smithi047.stdout:Installing dependencies: 2024-09-10T18:51:46.841 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-grafana-dashboards noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 24 k 2024-09-10T18:51:46.841 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-prometheus-alerts noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 15 k 2024-09-10T18:51:46.841 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-09-10T18:51:46.841 INFO:teuthology.orchestra.run.smithi047.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-10T18:51:46.841 INFO:teuthology.orchestra.run.smithi047.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-10T18:51:46.841 INFO:teuthology.orchestra.run.smithi047.stdout:Installing weak dependencies: 2024-09-10T18:51:46.841 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-09-10T18:51:46.841 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:51:46.841 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction Summary 2024-09-10T18:51:46.841 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-10T18:51:46.842 INFO:teuthology.orchestra.run.smithi047.stdout:Install 7 Packages 2024-09-10T18:51:46.842 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:51:46.842 INFO:teuthology.orchestra.run.smithi047.stdout:Total download size: 3.7 M 2024-09-10T18:51:46.842 INFO:teuthology.orchestra.run.smithi047.stdout:Installed size: 80 M 2024-09-10T18:51:46.842 INFO:teuthology.orchestra.run.smithi047.stdout:Downloading Packages: 2024-09-10T18:51:47.021 INFO:teuthology.orchestra.run.smithi047.stdout:(1/7): ceph-prometheus-alerts-17.2.7-1606.g5be5 99 kB/s | 15 kB 00:00 2024-09-10T18:51:47.038 INFO:teuthology.orchestra.run.smithi047.stdout:(2/7): ceph-grafana-dashboards-17.2.7-1606.g5be 148 kB/s | 24 kB 00:00 2024-09-10T18:51:47.080 INFO:teuthology.orchestra.run.smithi047.stdout:(3/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 154 kB/s | 9.0 kB 00:00 2024-09-10T18:51:47.106 INFO:teuthology.orchestra.run.smithi047.stdout:(4/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 1.3 MB/s | 31 kB 00:00 2024-09-10T18:51:47.131 INFO:teuthology.orchestra.run.smithi047.stdout:(5/7): python3-jwt-2.4.0-1.el9.noarch.rpm 442 kB/s | 41 kB 00:00 2024-09-10T18:51:47.156 INFO:teuthology.orchestra.run.smithi047.stdout:(6/7): python3-routes-2.5.1-5.el9.noarch.rpm 3.7 MB/s | 188 kB 00:00 2024-09-10T18:51:47.257 INFO:teuthology.orchestra.run.smithi047.stdout:(7/7): ceph-mgr-dashboard-17.2.7-1606.g5be57395 8.7 MB/s | 3.4 MB 00:00 2024-09-10T18:51:47.260 INFO:teuthology.orchestra.run.smithi047.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:51:47.261 INFO:teuthology.orchestra.run.smithi047.stdout:Total 8.7 MB/s | 3.7 MB 00:00 2024-09-10T18:51:47.261 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction check 2024-09-10T18:51:47.270 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction check succeeded. 2024-09-10T18:51:47.270 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction test 2024-09-10T18:51:47.374 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction test succeeded. 2024-09-10T18:51:47.375 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction 2024-09-10T18:51:47.622 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-1606.g5be57395.el9.noarc 7/7 2024-09-10T18:51:47.622 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-1606.g5be57395.el9. 1/7 2024-09-10T18:51:47.622 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-1606.g5be57395.el9.noarc 2/7 2024-09-10T18:51:47.622 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-1606.g5be57395.el9.n 3/7 2024-09-10T18:51:47.622 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-09-10T18:51:47.622 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-09-10T18:51:47.623 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-09-10T18:51:47.689 INFO:teuthology.orchestra.run.smithi047.stdout: Preparing : 1/1 2024-09-10T18:51:47.766 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-09-10T18:51:47.810 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-09-10T18:51:47.865 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-09-10T18:51:47.896 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-09-10T18:51:47.918 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-09-10T18:51:47.918 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T18:51:47.918 INFO:teuthology.orchestra.run.smithi149.stdout:Installed: 2024-09-10T18:51:47.918 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-grafana-dashboards-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:51:47.918 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-dashboard-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:51:47.918 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-prometheus-alerts-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:51:47.918 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-09-10T18:51:47.918 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-09-10T18:51:47.918 INFO:teuthology.orchestra.run.smithi149.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-10T18:51:47.918 INFO:teuthology.orchestra.run.smithi149.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-10T18:51:47.918 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T18:51:47.918 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-09-10T18:51:47.924 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-1606.g5be57395.el9.n 5/7 2024-09-10T18:51:48.135 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-10T18:51:48.703 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:01:00 ago on Tue 10 Sep 2024 06:50:48 PM UTC. 2024-09-10T18:51:48.837 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-09-10T18:51:48.837 INFO:teuthology.orchestra.run.smithi149.stdout:======================================================================================= 2024-09-10T18:51:48.838 INFO:teuthology.orchestra.run.smithi149.stdout: Package Arch Version Repository Size 2024-09-10T18:51:48.838 INFO:teuthology.orchestra.run.smithi149.stdout:======================================================================================= 2024-09-10T18:51:48.838 INFO:teuthology.orchestra.run.smithi149.stdout:Installing: 2024-09-10T18:51:48.838 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 7.4 M 2024-09-10T18:51:48.838 INFO:teuthology.orchestra.run.smithi149.stdout:Installing dependencies: 2024-09-10T18:51:48.838 INFO:teuthology.orchestra.run.smithi149.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-10T18:51:48.838 INFO:teuthology.orchestra.run.smithi149.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-10T18:51:48.838 INFO:teuthology.orchestra.run.smithi149.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-10T18:51:48.838 INFO:teuthology.orchestra.run.smithi149.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-10T18:51:48.838 INFO:teuthology.orchestra.run.smithi149.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-10T18:51:48.838 INFO:teuthology.orchestra.run.smithi149.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-10T18:51:48.838 INFO:teuthology.orchestra.run.smithi149.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-10T18:51:48.839 INFO:teuthology.orchestra.run.smithi149.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-10T18:51:48.839 INFO:teuthology.orchestra.run.smithi149.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-10T18:51:48.839 INFO:teuthology.orchestra.run.smithi149.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-10T18:51:48.839 INFO:teuthology.orchestra.run.smithi149.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-10T18:51:48.839 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T18:51:48.839 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction Summary 2024-09-10T18:51:48.839 INFO:teuthology.orchestra.run.smithi149.stdout:======================================================================================= 2024-09-10T18:51:48.839 INFO:teuthology.orchestra.run.smithi149.stdout:Install 12 Packages 2024-09-10T18:51:48.839 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T18:51:48.840 INFO:teuthology.orchestra.run.smithi149.stdout:Total download size: 38 M 2024-09-10T18:51:48.840 INFO:teuthology.orchestra.run.smithi149.stdout:Installed size: 212 M 2024-09-10T18:51:48.840 INFO:teuthology.orchestra.run.smithi149.stdout:Downloading Packages: 2024-09-10T18:51:48.984 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-1606.g5be57395.el9. 6/7 2024-09-10T18:51:49.019 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-1606.g5be57395.el9.noarc 7/7 2024-09-10T18:51:49.189 INFO:teuthology.orchestra.run.smithi149.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 899 kB/s | 191 kB 00:00 2024-09-10T18:51:49.315 INFO:teuthology.orchestra.run.smithi149.stdout:(2/12): flexiblas-3.0.4-8.el9.x86_64.rpm 261 kB/s | 33 kB 00:00 2024-09-10T18:51:49.431 INFO:teuthology.orchestra.run.smithi149.stdout:(3/12): ceph-mgr-diskprediction-local-17.2.7-16 16 MB/s | 7.4 MB 00:00 2024-09-10T18:51:49.457 INFO:teuthology.orchestra.run.smithi149.stdout:(4/12): libgfortran-11.5.0-2.el9.x86_64.rpm 1.6 MB/s | 801 kB 00:00 2024-09-10T18:51:49.552 INFO:teuthology.orchestra.run.smithi149.stdout:(5/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 155 kB/s | 18 kB 00:00 2024-09-10T18:51:49.574 INFO:teuthology.orchestra.run.smithi149.stdout:(6/12): openblas-0.3.26-2.el9.x86_64.rpm 328 kB/s | 38 kB 00:00 2024-09-10T18:51:49.699 INFO:teuthology.orchestra.run.smithi149.stdout:(7/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 7.8 MB/s | 3.0 MB 00:00 2024-09-10T18:51:49.758 INFO:teuthology.orchestra.run.smithi149.stdout:(8/12): python3-devel-3.9.19-8.el9.x86_64.rpm 1.3 MB/s | 245 kB 00:00 2024-09-10T18:51:49.842 INFO:teuthology.orchestra.run.smithi149.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.9 MB/s | 250 kB 00:00 2024-09-10T18:51:49.969 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-1606.g5be57395.el9.noarc 7/7 2024-09-10T18:51:49.970 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-1606.g5be57395.el9. 1/7 2024-09-10T18:51:49.970 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-1606.g5be57395.el9.noarc 2/7 2024-09-10T18:51:49.970 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-1606.g5be57395.el9.n 3/7 2024-09-10T18:51:49.970 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-09-10T18:51:49.970 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-09-10T18:51:49.970 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-09-10T18:51:50.000 INFO:teuthology.orchestra.run.smithi149.stdout:(10/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 17 MB/s | 5.1 MB 00:00 2024-09-10T18:51:50.067 INFO:teuthology.orchestra.run.smithi149.stdout:(11/12): openblas-openmp-0.3.26-2.el9.x86_64.rp 9.5 MB/s | 4.9 MB 00:00 2024-09-10T18:51:50.241 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-09-10T18:51:50.241 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:51:50.241 INFO:teuthology.orchestra.run.smithi047.stdout:Installed: 2024-09-10T18:51:50.241 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-grafana-dashboards-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:51:50.242 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-dashboard-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:51:50.242 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-prometheus-alerts-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:51:50.242 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-09-10T18:51:50.242 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-09-10T18:51:50.242 INFO:teuthology.orchestra.run.smithi047.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-10T18:51:50.242 INFO:teuthology.orchestra.run.smithi047.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-10T18:51:50.242 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:51:50.242 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-09-10T18:51:50.453 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-10T18:51:50.659 INFO:teuthology.orchestra.run.smithi149.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 20 MB/s | 16 MB 00:00 2024-09-10T18:51:50.660 INFO:teuthology.orchestra.run.smithi149.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:51:50.660 INFO:teuthology.orchestra.run.smithi149.stdout:Total 21 MB/s | 38 MB 00:01 2024-09-10T18:51:51.010 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction check 2024-09-10T18:51:51.036 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction check succeeded. 2024-09-10T18:51:51.036 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction test 2024-09-10T18:51:51.038 INFO:teuthology.orchestra.run.smithi047.stdout:Last metadata expiration check: 0:01:04 ago on Tue 10 Sep 2024 06:50:47 PM UTC. 2024-09-10T18:51:51.178 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-09-10T18:51:51.179 INFO:teuthology.orchestra.run.smithi047.stdout:======================================================================================= 2024-09-10T18:51:51.179 INFO:teuthology.orchestra.run.smithi047.stdout: Package Arch Version Repository Size 2024-09-10T18:51:51.179 INFO:teuthology.orchestra.run.smithi047.stdout:======================================================================================= 2024-09-10T18:51:51.179 INFO:teuthology.orchestra.run.smithi047.stdout:Installing: 2024-09-10T18:51:51.179 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 7.4 M 2024-09-10T18:51:51.179 INFO:teuthology.orchestra.run.smithi047.stdout:Installing dependencies: 2024-09-10T18:51:51.179 INFO:teuthology.orchestra.run.smithi047.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-10T18:51:51.179 INFO:teuthology.orchestra.run.smithi047.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-10T18:51:51.180 INFO:teuthology.orchestra.run.smithi047.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-10T18:51:51.180 INFO:teuthology.orchestra.run.smithi047.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-10T18:51:51.180 INFO:teuthology.orchestra.run.smithi047.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-10T18:51:51.180 INFO:teuthology.orchestra.run.smithi047.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-10T18:51:51.180 INFO:teuthology.orchestra.run.smithi047.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-10T18:51:51.180 INFO:teuthology.orchestra.run.smithi047.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-10T18:51:51.180 INFO:teuthology.orchestra.run.smithi047.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-10T18:51:51.180 INFO:teuthology.orchestra.run.smithi047.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-10T18:51:51.180 INFO:teuthology.orchestra.run.smithi047.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-10T18:51:51.180 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:51:51.180 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction Summary 2024-09-10T18:51:51.180 INFO:teuthology.orchestra.run.smithi047.stdout:======================================================================================= 2024-09-10T18:51:51.180 INFO:teuthology.orchestra.run.smithi047.stdout:Install 12 Packages 2024-09-10T18:51:51.180 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:51:51.181 INFO:teuthology.orchestra.run.smithi047.stdout:Total download size: 38 M 2024-09-10T18:51:51.181 INFO:teuthology.orchestra.run.smithi047.stdout:Installed size: 212 M 2024-09-10T18:51:51.181 INFO:teuthology.orchestra.run.smithi047.stdout:Downloading Packages: 2024-09-10T18:51:51.548 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction test succeeded. 2024-09-10T18:51:51.548 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction 2024-09-10T18:51:51.632 INFO:teuthology.orchestra.run.smithi047.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 618 kB/s | 191 kB 00:00 2024-09-10T18:51:51.724 INFO:teuthology.orchestra.run.smithi047.stdout:(2/12): libgfortran-11.5.0-2.el9.x86_64.rpm 1.9 MB/s | 801 kB 00:00 2024-09-10T18:51:51.749 INFO:teuthology.orchestra.run.smithi047.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 280 kB/s | 33 kB 00:00 2024-09-10T18:51:51.874 INFO:teuthology.orchestra.run.smithi047.stdout:(4/12): ceph-mgr-diskprediction-local-17.2.7-16 13 MB/s | 7.4 MB 00:00 2024-09-10T18:51:51.899 INFO:teuthology.orchestra.run.smithi047.stdout:(5/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 121 kB/s | 18 kB 00:00 2024-09-10T18:51:52.008 INFO:teuthology.orchestra.run.smithi047.stdout:(6/12): openblas-0.3.26-2.el9.x86_64.rpm 288 kB/s | 38 kB 00:00 2024-09-10T18:51:52.117 INFO:teuthology.orchestra.run.smithi047.stdout:(7/12): openblas-openmp-0.3.26-2.el9.x86_64.rpm 23 MB/s | 4.9 MB 00:00 2024-09-10T18:51:52.142 INFO:teuthology.orchestra.run.smithi047.stdout:(8/12): python3-devel-3.9.19-8.el9.x86_64.rpm 1.8 MB/s | 245 kB 00:00 2024-09-10T18:51:52.259 INFO:teuthology.orchestra.run.smithi047.stdout:(9/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 36 MB/s | 5.1 MB 00:00 2024-09-10T18:51:52.284 INFO:teuthology.orchestra.run.smithi047.stdout:(10/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 1.7 MB/s | 250 kB 00:00 2024-09-10T18:51:52.307 INFO:teuthology.orchestra.run.smithi149.stdout: Preparing : 1/1 2024-09-10T18:51:52.334 INFO:teuthology.orchestra.run.smithi047.stdout:(11/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rp 4.9 MB/s | 3.0 MB 00:00 2024-09-10T18:51:52.385 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-10T18:51:52.434 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-10T18:51:52.532 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-10T18:51:52.567 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-10T18:51:52.602 INFO:teuthology.orchestra.run.smithi047.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 48 MB/s | 16 MB 00:00 2024-09-10T18:51:52.602 INFO:teuthology.orchestra.run.smithi047.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:51:52.603 INFO:teuthology.orchestra.run.smithi047.stdout:Total 27 MB/s | 38 MB 00:01 2024-09-10T18:51:52.752 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-10T18:51:52.782 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-10T18:51:52.869 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-10T18:51:52.973 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction check 2024-09-10T18:51:53.000 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction check succeeded. 2024-09-10T18:51:53.000 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction test 2024-09-10T18:51:53.512 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-10T18:51:53.571 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction test succeeded. 2024-09-10T18:51:53.571 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction 2024-09-10T18:51:53.637 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-10T18:51:54.361 INFO:teuthology.orchestra.run.smithi047.stdout: Preparing : 1/1 2024-09-10T18:51:54.421 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-10T18:51:54.476 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-10T18:51:54.579 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-10T18:51:54.627 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-10T18:51:54.819 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-10T18:51:54.850 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-10T18:51:54.905 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-10T18:51:54.940 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-10T18:51:55.634 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-10T18:51:55.768 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-10T18:51:55.929 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-10T18:51:55.957 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-1606.g5be57 12/12 2024-09-10T18:51:57.081 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-10T18:51:58.170 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-10T18:51:58.208 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-1606.g5be57 12/12 2024-09-10T18:51:58.541 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-1606.g5be57 12/12 2024-09-10T18:51:58.541 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-1606.g5be57 1/12 2024-09-10T18:51:58.541 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-10T18:51:58.541 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-10T18:51:58.541 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-10T18:51:58.541 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-10T18:51:58.541 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-10T18:51:58.541 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-10T18:51:58.541 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-10T18:51:58.542 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-10T18:51:58.542 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-10T18:51:58.542 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-10T18:51:58.966 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-10T18:51:58.966 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T18:51:58.966 INFO:teuthology.orchestra.run.smithi149.stdout:Installed: 2024-09-10T18:51:58.966 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-diskprediction-local-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:51:58.966 INFO:teuthology.orchestra.run.smithi149.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-10T18:51:58.967 INFO:teuthology.orchestra.run.smithi149.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-10T18:51:58.967 INFO:teuthology.orchestra.run.smithi149.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-10T18:51:58.968 INFO:teuthology.orchestra.run.smithi149.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-10T18:51:58.968 INFO:teuthology.orchestra.run.smithi149.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-10T18:51:58.968 INFO:teuthology.orchestra.run.smithi149.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-10T18:51:58.968 INFO:teuthology.orchestra.run.smithi149.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-10T18:51:58.968 INFO:teuthology.orchestra.run.smithi149.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-10T18:51:58.968 INFO:teuthology.orchestra.run.smithi149.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-10T18:51:58.968 INFO:teuthology.orchestra.run.smithi149.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-10T18:51:58.968 INFO:teuthology.orchestra.run.smithi149.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-10T18:51:58.968 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T18:51:58.968 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-09-10T18:51:59.368 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install ceph-mgr-rook 2024-09-10T18:51:59.952 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:01:11 ago on Tue 10 Sep 2024 06:50:48 PM UTC. 2024-09-10T18:52:00.087 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-09-10T18:52:00.089 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-09-10T18:52:00.089 INFO:teuthology.orchestra.run.smithi149.stdout: Package Arch Version Repository Size 2024-09-10T18:52:00.089 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-09-10T18:52:00.089 INFO:teuthology.orchestra.run.smithi149.stdout:Installing: 2024-09-10T18:52:00.089 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-rook noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 47 k 2024-09-10T18:52:00.089 INFO:teuthology.orchestra.run.smithi149.stdout:Installing dependencies: 2024-09-10T18:52:00.089 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-10T18:52:00.089 INFO:teuthology.orchestra.run.smithi149.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-10T18:52:00.089 INFO:teuthology.orchestra.run.smithi149.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-10T18:52:00.089 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-10T18:52:00.089 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-10T18:52:00.090 INFO:teuthology.orchestra.run.smithi149.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-10T18:52:00.090 INFO:teuthology.orchestra.run.smithi149.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-10T18:52:00.090 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-10T18:52:00.090 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-10T18:52:00.090 INFO:teuthology.orchestra.run.smithi149.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-10T18:52:00.090 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-10T18:52:00.090 INFO:teuthology.orchestra.run.smithi149.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-10T18:52:00.090 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T18:52:00.090 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction Summary 2024-09-10T18:52:00.090 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-09-10T18:52:00.090 INFO:teuthology.orchestra.run.smithi149.stdout:Install 13 Packages 2024-09-10T18:52:00.090 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T18:52:00.091 INFO:teuthology.orchestra.run.smithi149.stdout:Total download size: 2.2 M 2024-09-10T18:52:00.091 INFO:teuthology.orchestra.run.smithi149.stdout:Installed size: 26 M 2024-09-10T18:52:00.091 INFO:teuthology.orchestra.run.smithi149.stdout:Downloading Packages: 2024-09-10T18:52:00.474 INFO:teuthology.orchestra.run.smithi149.stdout:(1/13): python3-jsonpointer-2.0-4.el9.noarch.rp 227 kB/s | 19 kB 00:00 2024-09-10T18:52:00.499 INFO:teuthology.orchestra.run.smithi149.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 234 kB/s | 26 kB 00:00 2024-09-10T18:52:00.558 INFO:teuthology.orchestra.run.smithi149.stdout:(3/13): ceph-mgr-rook-17.2.7-1606.g5be57395.el9 282 kB/s | 47 kB 00:00 2024-09-10T18:52:00.583 INFO:teuthology.orchestra.run.smithi149.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.0 MB/s | 222 kB 00:00 2024-09-10T18:52:00.634 INFO:teuthology.orchestra.run.smithi149.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.2 MB/s | 159 kB 00:00 2024-09-10T18:52:00.659 INFO:teuthology.orchestra.run.smithi149.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 714 kB/s | 54 kB 00:00 2024-09-10T18:52:00.701 INFO:teuthology.orchestra.run.smithi149.stdout:(7/13): python3-cachetools-4.2.4-1.el9.noarch.r 481 kB/s | 32 kB 00:00 2024-09-10T18:52:00.722 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-1606.g5be57 12/12 2024-09-10T18:52:00.723 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-1606.g5be57 1/12 2024-09-10T18:52:00.723 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-10T18:52:00.723 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-10T18:52:00.723 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-10T18:52:00.723 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-10T18:52:00.723 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-10T18:52:00.723 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-10T18:52:00.723 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-10T18:52:00.723 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-10T18:52:00.723 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-10T18:52:00.723 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-10T18:52:00.726 INFO:teuthology.orchestra.run.smithi149.stdout:(8/13): python3-certifi-2023.05.07-4.el9.noarch 211 kB/s | 14 kB 00:00 2024-09-10T18:52:00.751 INFO:teuthology.orchestra.run.smithi149.stdout:(9/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.4 MB/s | 279 kB 00:00 2024-09-10T18:52:00.777 INFO:teuthology.orchestra.run.smithi149.stdout:(10/13): python3-google-auth-2.34.0-1.el9.noarc 2.9 MB/s | 220 kB 00:00 2024-09-10T18:52:00.810 INFO:teuthology.orchestra.run.smithi149.stdout:(11/13): python3-websocket-client-1.2.3-2.el9.n 3.3 MB/s | 90 kB 00:00 2024-09-10T18:52:00.836 INFO:teuthology.orchestra.run.smithi149.stdout:(12/13): python3-rsa-4.9-2.el9.noarch.rpm 704 kB/s | 59 kB 00:00 2024-09-10T18:52:00.869 INFO:teuthology.orchestra.run.smithi149.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 7.2 MB/s | 1.0 MB 00:00 2024-09-10T18:52:00.872 INFO:teuthology.orchestra.run.smithi149.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:52:00.872 INFO:teuthology.orchestra.run.smithi149.stdout:Total 2.8 MB/s | 2.2 MB 00:00 2024-09-10T18:52:00.935 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction check 2024-09-10T18:52:00.950 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction check succeeded. 2024-09-10T18:52:00.951 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction test 2024-09-10T18:52:01.040 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction test succeeded. 2024-09-10T18:52:01.042 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction 2024-09-10T18:52:01.182 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-10T18:52:01.183 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:52:01.183 INFO:teuthology.orchestra.run.smithi047.stdout:Installed: 2024-09-10T18:52:01.183 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-diskprediction-local-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:52:01.183 INFO:teuthology.orchestra.run.smithi047.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-10T18:52:01.183 INFO:teuthology.orchestra.run.smithi047.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-10T18:52:01.183 INFO:teuthology.orchestra.run.smithi047.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-10T18:52:01.183 INFO:teuthology.orchestra.run.smithi047.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-10T18:52:01.183 INFO:teuthology.orchestra.run.smithi047.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-10T18:52:01.183 INFO:teuthology.orchestra.run.smithi047.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-10T18:52:01.183 INFO:teuthology.orchestra.run.smithi047.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-10T18:52:01.183 INFO:teuthology.orchestra.run.smithi047.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-10T18:52:01.184 INFO:teuthology.orchestra.run.smithi047.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-10T18:52:01.184 INFO:teuthology.orchestra.run.smithi047.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-10T18:52:01.184 INFO:teuthology.orchestra.run.smithi047.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-10T18:52:01.184 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:52:01.184 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-09-10T18:52:01.539 INFO:teuthology.orchestra.run.smithi149.stdout: Preparing : 1/1 2024-09-10T18:52:01.684 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install ceph-mgr-rook 2024-09-10T18:52:01.735 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-10T18:52:01.786 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-10T18:52:01.842 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-10T18:52:01.888 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-10T18:52:01.929 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-10T18:52:02.040 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-10T18:52:02.149 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-10T18:52:02.198 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-10T18:52:02.285 INFO:teuthology.orchestra.run.smithi047.stdout:Last metadata expiration check: 0:01:15 ago on Tue 10 Sep 2024 06:50:47 PM UTC. 2024-09-10T18:52:02.425 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-09-10T18:52:02.427 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-10T18:52:02.427 INFO:teuthology.orchestra.run.smithi047.stdout: Package Arch Version Repository Size 2024-09-10T18:52:02.427 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-10T18:52:02.427 INFO:teuthology.orchestra.run.smithi047.stdout:Installing: 2024-09-10T18:52:02.427 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-rook noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 47 k 2024-09-10T18:52:02.427 INFO:teuthology.orchestra.run.smithi047.stdout:Installing dependencies: 2024-09-10T18:52:02.428 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-10T18:52:02.428 INFO:teuthology.orchestra.run.smithi047.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-10T18:52:02.428 INFO:teuthology.orchestra.run.smithi047.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-10T18:52:02.428 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-10T18:52:02.428 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-10T18:52:02.429 INFO:teuthology.orchestra.run.smithi047.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-10T18:52:02.429 INFO:teuthology.orchestra.run.smithi047.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-10T18:52:02.429 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-10T18:52:02.429 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-10T18:52:02.429 INFO:teuthology.orchestra.run.smithi047.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-10T18:52:02.430 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-10T18:52:02.430 INFO:teuthology.orchestra.run.smithi047.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-10T18:52:02.430 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:52:02.430 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction Summary 2024-09-10T18:52:02.430 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-10T18:52:02.430 INFO:teuthology.orchestra.run.smithi047.stdout:Install 13 Packages 2024-09-10T18:52:02.431 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:52:02.432 INFO:teuthology.orchestra.run.smithi047.stdout:Total download size: 2.2 M 2024-09-10T18:52:02.432 INFO:teuthology.orchestra.run.smithi047.stdout:Installed size: 26 M 2024-09-10T18:52:02.432 INFO:teuthology.orchestra.run.smithi047.stdout:Downloading Packages: 2024-09-10T18:52:02.835 INFO:teuthology.orchestra.run.smithi047.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 270 kB/s | 26 kB 00:00 2024-09-10T18:52:02.860 INFO:teuthology.orchestra.run.smithi047.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 160 kB/s | 19 kB 00:00 2024-09-10T18:52:02.918 INFO:teuthology.orchestra.run.smithi047.stdout:(3/13): ceph-mgr-rook-17.2.7-1606.g5be57395.el9 266 kB/s | 47 kB 00:00 2024-09-10T18:52:02.920 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-10T18:52:02.944 INFO:teuthology.orchestra.run.smithi047.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.0 MB/s | 222 kB 00:00 2024-09-10T18:52:02.970 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-10T18:52:02.986 INFO:teuthology.orchestra.run.smithi047.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.2 MB/s | 159 kB 00:00 2024-09-10T18:52:03.004 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-10T18:52:03.028 INFO:teuthology.orchestra.run.smithi047.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 640 kB/s | 54 kB 00:00 2024-09-10T18:52:03.049 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-10T18:52:03.061 INFO:teuthology.orchestra.run.smithi047.stdout:(7/13): python3-cachetools-4.2.4-1.el9.noarch.r 427 kB/s | 32 kB 00:00 2024-09-10T18:52:03.074 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-rook-2:17.2.7-1606.g5be57395.el9.noarch 13/13 2024-09-10T18:52:03.103 INFO:teuthology.orchestra.run.smithi047.stdout:(8/13): python3-certifi-2023.05.07-4.el9.noarch 193 kB/s | 14 kB 00:00 2024-09-10T18:52:03.128 INFO:teuthology.orchestra.run.smithi047.stdout:(9/13): python3-google-auth-2.34.0-1.el9.noarch 3.3 MB/s | 220 kB 00:00 2024-09-10T18:52:03.154 INFO:teuthology.orchestra.run.smithi047.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-10T18:52:03.179 INFO:teuthology.orchestra.run.smithi047.stdout:(11/13): python3-pyasn1-modules-0.4.8-6.el9.noa 1.0 MB/s | 279 kB 00:00 2024-09-10T18:52:03.204 INFO:teuthology.orchestra.run.smithi047.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.8 MB/s | 90 kB 00:00 2024-09-10T18:52:03.238 INFO:teuthology.orchestra.run.smithi047.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 7.6 MB/s | 1.0 MB 00:00 2024-09-10T18:52:03.240 INFO:teuthology.orchestra.run.smithi047.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:52:03.240 INFO:teuthology.orchestra.run.smithi047.stdout:Total 2.7 MB/s | 2.2 MB 00:00 2024-09-10T18:52:03.310 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction check 2024-09-10T18:52:03.326 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction check succeeded. 2024-09-10T18:52:03.326 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction test 2024-09-10T18:52:03.422 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction test succeeded. 2024-09-10T18:52:03.422 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction 2024-09-10T18:52:03.872 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-1606.g5be57395.el9.noarch 13/13 2024-09-10T18:52:03.873 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-rook-2:17.2.7-1606.g5be57395.el9.noarch 1/13 2024-09-10T18:52:03.873 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-10T18:52:03.873 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-10T18:52:03.873 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-10T18:52:03.873 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-10T18:52:03.873 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-10T18:52:03.873 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-10T18:52:03.873 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-10T18:52:03.873 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-10T18:52:03.873 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-10T18:52:03.873 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-10T18:52:03.873 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-10T18:52:04.067 INFO:teuthology.orchestra.run.smithi047.stdout: Preparing : 1/1 2024-09-10T18:52:04.216 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-10T18:52:04.217 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T18:52:04.217 INFO:teuthology.orchestra.run.smithi149.stdout:Installed: 2024-09-10T18:52:04.217 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-rook-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:52:04.217 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-10T18:52:04.217 INFO:teuthology.orchestra.run.smithi149.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-10T18:52:04.217 INFO:teuthology.orchestra.run.smithi149.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-10T18:52:04.217 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-10T18:52:04.217 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-10T18:52:04.217 INFO:teuthology.orchestra.run.smithi149.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-10T18:52:04.217 INFO:teuthology.orchestra.run.smithi149.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-10T18:52:04.217 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-10T18:52:04.217 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-10T18:52:04.217 INFO:teuthology.orchestra.run.smithi149.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-10T18:52:04.218 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-10T18:52:04.218 INFO:teuthology.orchestra.run.smithi149.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-10T18:52:04.218 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T18:52:04.218 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-09-10T18:52:04.235 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-10T18:52:04.292 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-10T18:52:04.348 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-10T18:52:04.382 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-10T18:52:04.425 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-10T18:52:04.468 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install ceph-mgr-cephadm 2024-09-10T18:52:04.532 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-10T18:52:04.642 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-10T18:52:04.721 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-10T18:52:05.053 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:01:17 ago on Tue 10 Sep 2024 06:50:48 PM UTC. 2024-09-10T18:52:05.187 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-09-10T18:52:05.187 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-09-10T18:52:05.187 INFO:teuthology.orchestra.run.smithi149.stdout: Package Arch Version Repository Size 2024-09-10T18:52:05.187 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-09-10T18:52:05.188 INFO:teuthology.orchestra.run.smithi149.stdout:Installing: 2024-09-10T18:52:05.188 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-cephadm noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 124 k 2024-09-10T18:52:05.188 INFO:teuthology.orchestra.run.smithi149.stdout:Installing dependencies: 2024-09-10T18:52:05.188 INFO:teuthology.orchestra.run.smithi149.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-10T18:52:05.188 INFO:teuthology.orchestra.run.smithi149.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-10T18:52:05.188 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-10T18:52:05.188 INFO:teuthology.orchestra.run.smithi149.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-10T18:52:05.188 INFO:teuthology.orchestra.run.smithi149.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-10T18:52:05.188 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T18:52:05.188 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction Summary 2024-09-10T18:52:05.188 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-09-10T18:52:05.188 INFO:teuthology.orchestra.run.smithi149.stdout:Install 6 Packages 2024-09-10T18:52:05.188 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T18:52:05.189 INFO:teuthology.orchestra.run.smithi149.stdout:Total download size: 7.0 M 2024-09-10T18:52:05.189 INFO:teuthology.orchestra.run.smithi149.stdout:Installed size: 33 M 2024-09-10T18:52:05.189 INFO:teuthology.orchestra.run.smithi149.stdout:Downloading Packages: 2024-09-10T18:52:05.397 INFO:teuthology.orchestra.run.smithi149.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.9 MB/s | 248 kB 00:00 2024-09-10T18:52:05.472 INFO:teuthology.orchestra.run.smithi149.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 7.2 MB/s | 548 kB 00:00 2024-09-10T18:52:05.498 INFO:teuthology.orchestra.run.smithi149.stdout:(3/6): ceph-mgr-cephadm-17.2.7-1606.g5be57395.e 545 kB/s | 124 kB 00:00 2024-09-10T18:52:05.498 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-10T18:52:05.524 INFO:teuthology.orchestra.run.smithi149.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 1.1 MB/s | 58 kB 00:00 2024-09-10T18:52:05.539 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-10T18:52:05.565 INFO:teuthology.orchestra.run.smithi149.stdout:(5/6): python3-typing-extensions-4.12.2-2.el9.n 1.1 MB/s | 74 kB 00:00 2024-09-10T18:52:05.573 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-10T18:52:05.620 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-10T18:52:05.642 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mgr-rook-2:17.2.7-1606.g5be57395.el9.noarch 13/13 2024-09-10T18:52:06.444 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-1606.g5be57395.el9.noarch 13/13 2024-09-10T18:52:06.444 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-rook-2:17.2.7-1606.g5be57395.el9.noarch 1/13 2024-09-10T18:52:06.445 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-10T18:52:06.445 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-10T18:52:06.445 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-10T18:52:06.445 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-10T18:52:06.445 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-10T18:52:06.445 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-10T18:52:06.445 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-10T18:52:06.445 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-10T18:52:06.445 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-10T18:52:06.445 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-10T18:52:06.445 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-10T18:52:06.699 INFO:teuthology.orchestra.run.smithi149.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 4.2 MB/s | 6.0 MB 00:01 2024-09-10T18:52:06.701 INFO:teuthology.orchestra.run.smithi149.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:52:06.701 INFO:teuthology.orchestra.run.smithi149.stdout:Total 4.6 MB/s | 7.0 MB 00:01 2024-09-10T18:52:06.772 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction check 2024-09-10T18:52:06.781 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-10T18:52:06.781 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:52:06.781 INFO:teuthology.orchestra.run.smithi047.stdout:Installed: 2024-09-10T18:52:06.781 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-rook-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:52:06.781 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-10T18:52:06.781 INFO:teuthology.orchestra.run.smithi047.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-10T18:52:06.781 INFO:teuthology.orchestra.run.smithi047.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-10T18:52:06.781 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-10T18:52:06.781 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-10T18:52:06.781 INFO:teuthology.orchestra.run.smithi047.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-10T18:52:06.781 INFO:teuthology.orchestra.run.smithi047.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-10T18:52:06.781 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-10T18:52:06.782 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-10T18:52:06.782 INFO:teuthology.orchestra.run.smithi047.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-10T18:52:06.782 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-10T18:52:06.782 INFO:teuthology.orchestra.run.smithi047.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-10T18:52:06.782 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:52:06.782 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-09-10T18:52:06.783 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction check succeeded. 2024-09-10T18:52:06.783 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction test 2024-09-10T18:52:06.885 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction test succeeded. 2024-09-10T18:52:06.885 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction 2024-09-10T18:52:07.172 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install ceph-mgr-cephadm 2024-09-10T18:52:07.193 INFO:teuthology.orchestra.run.smithi149.stdout: Preparing : 1/1 2024-09-10T18:52:07.372 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-10T18:52:07.426 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-10T18:52:07.756 INFO:teuthology.orchestra.run.smithi047.stdout:Last metadata expiration check: 0:01:20 ago on Tue 10 Sep 2024 06:50:47 PM UTC. 2024-09-10T18:52:07.896 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-09-10T18:52:07.896 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-10T18:52:07.896 INFO:teuthology.orchestra.run.smithi047.stdout: Package Arch Version Repository Size 2024-09-10T18:52:07.897 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-10T18:52:07.897 INFO:teuthology.orchestra.run.smithi047.stdout:Installing: 2024-09-10T18:52:07.897 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-cephadm noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 124 k 2024-09-10T18:52:07.897 INFO:teuthology.orchestra.run.smithi047.stdout:Installing dependencies: 2024-09-10T18:52:07.897 INFO:teuthology.orchestra.run.smithi047.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-10T18:52:07.897 INFO:teuthology.orchestra.run.smithi047.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-10T18:52:07.897 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-10T18:52:07.897 INFO:teuthology.orchestra.run.smithi047.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-10T18:52:07.897 INFO:teuthology.orchestra.run.smithi047.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-10T18:52:07.897 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:52:07.897 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction Summary 2024-09-10T18:52:07.897 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-10T18:52:07.898 INFO:teuthology.orchestra.run.smithi047.stdout:Install 6 Packages 2024-09-10T18:52:07.898 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:52:07.898 INFO:teuthology.orchestra.run.smithi047.stdout:Total download size: 7.0 M 2024-09-10T18:52:07.898 INFO:teuthology.orchestra.run.smithi047.stdout:Installed size: 33 M 2024-09-10T18:52:07.898 INFO:teuthology.orchestra.run.smithi047.stdout:Downloading Packages: 2024-09-10T18:52:07.951 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-10T18:52:08.037 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-10T18:52:08.100 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-10T18:52:08.129 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-1606.g5be57395.el9.noarch 6/6 2024-09-10T18:52:08.765 INFO:teuthology.orchestra.run.smithi047.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.4 MB/s | 248 kB 00:00 2024-09-10T18:52:08.790 INFO:teuthology.orchestra.run.smithi047.stdout:(2/6): ceph-mgr-cephadm-17.2.7-1606.g5be57395.e 623 kB/s | 124 kB 00:00 2024-09-10T18:52:08.832 INFO:teuthology.orchestra.run.smithi047.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 1.4 MB/s | 58 kB 00:00 2024-09-10T18:52:08.857 INFO:teuthology.orchestra.run.smithi047.stdout:(4/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 5.8 MB/s | 548 kB 00:00 2024-09-10T18:52:08.882 INFO:teuthology.orchestra.run.smithi047.stdout:(5/6): python3-typing-extensions-4.12.2-2.el9.n 1.5 MB/s | 74 kB 00:00 2024-09-10T18:52:08.920 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-1606.g5be57395.el9.noarch 6/6 2024-09-10T18:52:08.920 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-1606.g5be57395.el9.noarch 1/6 2024-09-10T18:52:08.920 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-10T18:52:08.921 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-10T18:52:08.921 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-10T18:52:08.921 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-10T18:52:08.999 INFO:teuthology.orchestra.run.smithi047.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 15 MB/s | 6.0 MB 00:00 2024-09-10T18:52:09.003 INFO:teuthology.orchestra.run.smithi047.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:52:09.003 INFO:teuthology.orchestra.run.smithi047.stdout:Total 6.3 MB/s | 7.0 MB 00:01 2024-09-10T18:52:09.079 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction check 2024-09-10T18:52:09.092 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction check succeeded. 2024-09-10T18:52:09.092 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction test 2024-09-10T18:52:09.198 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-10T18:52:09.198 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T18:52:09.198 INFO:teuthology.orchestra.run.smithi149.stdout:Installed: 2024-09-10T18:52:09.198 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-cephadm-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:52:09.198 INFO:teuthology.orchestra.run.smithi149.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-10T18:52:09.199 INFO:teuthology.orchestra.run.smithi149.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-10T18:52:09.199 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-10T18:52:09.199 INFO:teuthology.orchestra.run.smithi149.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-10T18:52:09.199 INFO:teuthology.orchestra.run.smithi149.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-10T18:52:09.199 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T18:52:09.199 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-09-10T18:52:09.200 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction test succeeded. 2024-09-10T18:52:09.200 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction 2024-09-10T18:52:09.472 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install ceph-fuse 2024-09-10T18:52:09.655 INFO:teuthology.orchestra.run.smithi047.stdout: Preparing : 1/1 2024-09-10T18:52:09.832 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-10T18:52:09.886 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-10T18:52:10.048 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:01:22 ago on Tue 10 Sep 2024 06:50:48 PM UTC. 2024-09-10T18:52:10.183 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-09-10T18:52:10.183 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-09-10T18:52:10.183 INFO:teuthology.orchestra.run.smithi149.stdout: Package Arch Version Repository Size 2024-09-10T18:52:10.183 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-09-10T18:52:10.183 INFO:teuthology.orchestra.run.smithi149.stdout:Installing: 2024-09-10T18:52:10.184 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-fuse x86_64 2:17.2.7-1606.g5be57395.el9 ceph 813 k 2024-09-10T18:52:10.184 INFO:teuthology.orchestra.run.smithi149.stdout:Installing dependencies: 2024-09-10T18:52:10.184 INFO:teuthology.orchestra.run.smithi149.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-10T18:52:10.184 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T18:52:10.184 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction Summary 2024-09-10T18:52:10.184 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-09-10T18:52:10.184 INFO:teuthology.orchestra.run.smithi149.stdout:Install 2 Packages 2024-09-10T18:52:10.184 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T18:52:10.185 INFO:teuthology.orchestra.run.smithi149.stdout:Total download size: 893 k 2024-09-10T18:52:10.185 INFO:teuthology.orchestra.run.smithi149.stdout:Installed size: 2.6 M 2024-09-10T18:52:10.185 INFO:teuthology.orchestra.run.smithi149.stdout:Downloading Packages: 2024-09-10T18:52:10.448 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-10T18:52:10.539 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-10T18:52:10.611 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-10T18:52:10.639 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-1606.g5be57395.el9.noarch 6/6 2024-09-10T18:52:10.726 INFO:teuthology.orchestra.run.smithi149.stdout:(1/2): ceph-fuse-17.2.7-1606.g5be57395.el9.x86_ 3.0 MB/s | 813 kB 00:00 2024-09-10T18:52:10.751 INFO:teuthology.orchestra.run.smithi149.stdout:(2/2): fuse-2.9.9-16.el9.x86_64.rpm 278 kB/s | 80 kB 00:00 2024-09-10T18:52:10.751 INFO:teuthology.orchestra.run.smithi149.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:52:10.752 INFO:teuthology.orchestra.run.smithi149.stdout:Total 1.5 MB/s | 893 kB 00:00 2024-09-10T18:52:10.764 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction check 2024-09-10T18:52:10.785 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction check succeeded. 2024-09-10T18:52:10.785 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction test 2024-09-10T18:52:10.865 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction test succeeded. 2024-09-10T18:52:10.866 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction 2024-09-10T18:52:11.061 INFO:teuthology.orchestra.run.smithi149.stdout: Preparing : 1/1 2024-09-10T18:52:11.196 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-10T18:52:11.259 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 2/2 2024-09-10T18:52:11.431 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-1606.g5be57395.el9.noarch 6/6 2024-09-10T18:52:11.431 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-1606.g5be57395.el9.noarch 1/6 2024-09-10T18:52:11.431 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-10T18:52:11.432 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-10T18:52:11.432 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-10T18:52:11.432 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-10T18:52:11.730 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-10T18:52:11.730 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:52:11.730 INFO:teuthology.orchestra.run.smithi047.stdout:Installed: 2024-09-10T18:52:11.730 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-cephadm-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:52:11.730 INFO:teuthology.orchestra.run.smithi047.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-10T18:52:11.730 INFO:teuthology.orchestra.run.smithi047.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-10T18:52:11.730 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-10T18:52:11.730 INFO:teuthology.orchestra.run.smithi047.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-10T18:52:11.730 INFO:teuthology.orchestra.run.smithi047.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-10T18:52:11.730 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:52:11.730 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-09-10T18:52:11.916 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 2/2 2024-09-10T18:52:11.916 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 1/2 2024-09-10T18:52:12.004 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install ceph-fuse 2024-09-10T18:52:12.196 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-10T18:52:12.196 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T18:52:12.196 INFO:teuthology.orchestra.run.smithi149.stdout:Installed: 2024-09-10T18:52:12.196 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-10T18:52:12.196 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T18:52:12.196 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-09-10T18:52:12.474 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install ceph-volume 2024-09-10T18:52:12.594 INFO:teuthology.orchestra.run.smithi047.stdout:Last metadata expiration check: 0:01:25 ago on Tue 10 Sep 2024 06:50:47 PM UTC. 2024-09-10T18:52:12.731 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-09-10T18:52:12.732 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-10T18:52:12.732 INFO:teuthology.orchestra.run.smithi047.stdout: Package Arch Version Repository Size 2024-09-10T18:52:12.732 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-10T18:52:12.733 INFO:teuthology.orchestra.run.smithi047.stdout:Installing: 2024-09-10T18:52:12.733 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-fuse x86_64 2:17.2.7-1606.g5be57395.el9 ceph 813 k 2024-09-10T18:52:12.733 INFO:teuthology.orchestra.run.smithi047.stdout:Installing dependencies: 2024-09-10T18:52:12.733 INFO:teuthology.orchestra.run.smithi047.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-10T18:52:12.733 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:52:12.733 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction Summary 2024-09-10T18:52:12.733 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-10T18:52:12.733 INFO:teuthology.orchestra.run.smithi047.stdout:Install 2 Packages 2024-09-10T18:52:12.733 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:52:12.733 INFO:teuthology.orchestra.run.smithi047.stdout:Total download size: 893 k 2024-09-10T18:52:12.733 INFO:teuthology.orchestra.run.smithi047.stdout:Installed size: 2.6 M 2024-09-10T18:52:12.733 INFO:teuthology.orchestra.run.smithi047.stdout:Downloading Packages: 2024-09-10T18:52:12.877 INFO:teuthology.orchestra.run.smithi047.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 658 kB/s | 80 kB 00:00 2024-09-10T18:52:13.043 INFO:teuthology.orchestra.run.smithi047.stdout:(2/2): ceph-fuse-17.2.7-1606.g5be57395.el9.x86_ 2.8 MB/s | 813 kB 00:00 2024-09-10T18:52:13.044 INFO:teuthology.orchestra.run.smithi047.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:52:13.044 INFO:teuthology.orchestra.run.smithi047.stdout:Total 2.8 MB/s | 893 kB 00:00 2024-09-10T18:52:13.054 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:01:25 ago on Tue 10 Sep 2024 06:50:48 PM UTC. 2024-09-10T18:52:13.056 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction check 2024-09-10T18:52:13.079 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction check succeeded. 2024-09-10T18:52:13.079 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction test 2024-09-10T18:52:13.163 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction test succeeded. 2024-09-10T18:52:13.164 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction 2024-09-10T18:52:13.189 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-09-10T18:52:13.189 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-09-10T18:52:13.190 INFO:teuthology.orchestra.run.smithi149.stdout: Package Arch Version Repository Size 2024-09-10T18:52:13.190 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-09-10T18:52:13.190 INFO:teuthology.orchestra.run.smithi149.stdout:Installing: 2024-09-10T18:52:13.190 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-volume noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 270 k 2024-09-10T18:52:13.190 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T18:52:13.190 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction Summary 2024-09-10T18:52:13.190 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-09-10T18:52:13.190 INFO:teuthology.orchestra.run.smithi149.stdout:Install 1 Package 2024-09-10T18:52:13.190 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T18:52:13.190 INFO:teuthology.orchestra.run.smithi149.stdout:Total download size: 270 k 2024-09-10T18:52:13.190 INFO:teuthology.orchestra.run.smithi149.stdout:Installed size: 1.3 M 2024-09-10T18:52:13.190 INFO:teuthology.orchestra.run.smithi149.stdout:Downloading Packages: 2024-09-10T18:52:13.363 INFO:teuthology.orchestra.run.smithi047.stdout: Preparing : 1/1 2024-09-10T18:52:13.423 INFO:teuthology.orchestra.run.smithi149.stdout:ceph-volume-17.2.7-1606.g5be57395.el9.noarch.rp 1.1 MB/s | 270 kB 00:00 2024-09-10T18:52:13.424 INFO:teuthology.orchestra.run.smithi149.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:52:13.424 INFO:teuthology.orchestra.run.smithi149.stdout:Total 1.1 MB/s | 270 kB 00:00 2024-09-10T18:52:13.425 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction check 2024-09-10T18:52:13.433 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction check succeeded. 2024-09-10T18:52:13.433 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction test 2024-09-10T18:52:13.466 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction test succeeded. 2024-09-10T18:52:13.466 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction 2024-09-10T18:52:13.510 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-10T18:52:13.587 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 2/2 2024-09-10T18:52:13.685 INFO:teuthology.orchestra.run.smithi149.stdout: Preparing : 1/1 2024-09-10T18:52:13.792 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-volume-2:17.2.7-1606.g5be57395.el9.noarch 1/1 2024-09-10T18:52:13.808 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-volume-2:17.2.7-1606.g5be57395.el9.noarch 1/1 2024-09-10T18:52:13.809 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T18:52:13.809 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-10T18:52:13.809 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T18:52:14.394 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 2/2 2024-09-10T18:52:14.394 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 1/2 2024-09-10T18:52:14.714 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-10T18:52:14.714 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:52:14.714 INFO:teuthology.orchestra.run.smithi047.stdout:Installed: 2024-09-10T18:52:14.714 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-10T18:52:14.714 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:52:14.714 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-09-10T18:52:14.771 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-volume-2:17.2.7-1606.g5be57395.el9.noarch 1/1 2024-09-10T18:52:14.771 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T18:52:14.771 INFO:teuthology.orchestra.run.smithi149.stdout:Installed: 2024-09-10T18:52:14.771 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-volume-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:52:14.771 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T18:52:14.771 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-09-10T18:52:15.003 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install ceph-volume 2024-09-10T18:52:15.087 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install librados-devel 2024-09-10T18:52:15.600 INFO:teuthology.orchestra.run.smithi047.stdout:Last metadata expiration check: 0:01:28 ago on Tue 10 Sep 2024 06:50:47 PM UTC. 2024-09-10T18:52:15.674 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:01:27 ago on Tue 10 Sep 2024 06:50:48 PM UTC. 2024-09-10T18:52:15.739 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-09-10T18:52:15.740 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-10T18:52:15.740 INFO:teuthology.orchestra.run.smithi047.stdout: Package Arch Version Repository Size 2024-09-10T18:52:15.740 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-10T18:52:15.740 INFO:teuthology.orchestra.run.smithi047.stdout:Installing: 2024-09-10T18:52:15.740 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-volume noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 270 k 2024-09-10T18:52:15.740 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:52:15.740 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction Summary 2024-09-10T18:52:15.740 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-10T18:52:15.741 INFO:teuthology.orchestra.run.smithi047.stdout:Install 1 Package 2024-09-10T18:52:15.741 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:52:15.741 INFO:teuthology.orchestra.run.smithi047.stdout:Total download size: 270 k 2024-09-10T18:52:15.741 INFO:teuthology.orchestra.run.smithi047.stdout:Installed size: 1.3 M 2024-09-10T18:52:15.741 INFO:teuthology.orchestra.run.smithi047.stdout:Downloading Packages: 2024-09-10T18:52:15.812 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-09-10T18:52:15.812 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-09-10T18:52:15.812 INFO:teuthology.orchestra.run.smithi149.stdout: Package Arch Version Repo Size 2024-09-10T18:52:15.812 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-09-10T18:52:15.812 INFO:teuthology.orchestra.run.smithi149.stdout:Installing: 2024-09-10T18:52:15.812 INFO:teuthology.orchestra.run.smithi149.stdout: librados-devel x86_64 2:17.2.7-1606.g5be57395.el9 ceph 126 k 2024-09-10T18:52:15.812 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T18:52:15.812 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction Summary 2024-09-10T18:52:15.813 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-09-10T18:52:15.813 INFO:teuthology.orchestra.run.smithi149.stdout:Install 1 Package 2024-09-10T18:52:15.813 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T18:52:15.813 INFO:teuthology.orchestra.run.smithi149.stdout:Total download size: 126 k 2024-09-10T18:52:15.813 INFO:teuthology.orchestra.run.smithi149.stdout:Installed size: 448 k 2024-09-10T18:52:15.813 INFO:teuthology.orchestra.run.smithi149.stdout:Downloading Packages: 2024-09-10T18:52:15.966 INFO:teuthology.orchestra.run.smithi047.stdout:ceph-volume-17.2.7-1606.g5be57395.el9.noarch.rp 1.2 MB/s | 270 kB 00:00 2024-09-10T18:52:15.966 INFO:teuthology.orchestra.run.smithi047.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:52:15.966 INFO:teuthology.orchestra.run.smithi047.stdout:Total 1.2 MB/s | 270 kB 00:00 2024-09-10T18:52:15.967 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction check 2024-09-10T18:52:15.976 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction check succeeded. 2024-09-10T18:52:15.976 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction test 2024-09-10T18:52:16.004 INFO:teuthology.orchestra.run.smithi149.stdout:librados-devel-17.2.7-1606.g5be57395.el9.x86_64 663 kB/s | 126 kB 00:00 2024-09-10T18:52:16.005 INFO:teuthology.orchestra.run.smithi149.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:52:16.005 INFO:teuthology.orchestra.run.smithi149.stdout:Total 658 kB/s | 126 kB 00:00 2024-09-10T18:52:16.005 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction check 2024-09-10T18:52:16.010 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction check succeeded. 2024-09-10T18:52:16.010 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction test 2024-09-10T18:52:16.013 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction test succeeded. 2024-09-10T18:52:16.013 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction 2024-09-10T18:52:16.074 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction test succeeded. 2024-09-10T18:52:16.074 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction 2024-09-10T18:52:16.230 INFO:teuthology.orchestra.run.smithi149.stdout: Preparing : 1/1 2024-09-10T18:52:16.238 INFO:teuthology.orchestra.run.smithi047.stdout: Preparing : 1/1 2024-09-10T18:52:16.301 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-volume-2:17.2.7-1606.g5be57395.el9.noarch 1/1 2024-09-10T18:52:16.318 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-volume-2:17.2.7-1606.g5be57395.el9.noarch 1/1 2024-09-10T18:52:16.318 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T18:52:16.318 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-10T18:52:16.318 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:52:16.327 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : librados-devel-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:52:16.795 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: librados-devel-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:52:17.038 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librados-devel-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:52:17.038 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T18:52:17.038 INFO:teuthology.orchestra.run.smithi149.stdout:Installed: 2024-09-10T18:52:17.038 INFO:teuthology.orchestra.run.smithi149.stdout: librados-devel-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:52:17.038 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T18:52:17.038 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-09-10T18:52:17.366 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install libcephfs2 2024-09-10T18:52:17.387 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-volume-2:17.2.7-1606.g5be57395.el9.noarch 1/1 2024-09-10T18:52:17.387 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:52:17.387 INFO:teuthology.orchestra.run.smithi047.stdout:Installed: 2024-09-10T18:52:17.387 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-volume-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:52:17.387 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:52:17.387 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-09-10T18:52:17.675 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install librados-devel 2024-09-10T18:52:17.952 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:01:29 ago on Tue 10 Sep 2024 06:50:48 PM UTC. 2024-09-10T18:52:18.046 INFO:teuthology.orchestra.run.smithi149.stdout:Package libcephfs2-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T18:52:18.091 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-09-10T18:52:18.092 INFO:teuthology.orchestra.run.smithi149.stdout:Nothing to do. 2024-09-10T18:52:18.092 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-09-10T18:52:18.182 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install libcephfs-devel 2024-09-10T18:52:18.275 INFO:teuthology.orchestra.run.smithi047.stdout:Last metadata expiration check: 0:01:31 ago on Tue 10 Sep 2024 06:50:47 PM UTC. 2024-09-10T18:52:18.415 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-09-10T18:52:18.416 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-10T18:52:18.416 INFO:teuthology.orchestra.run.smithi047.stdout: Package Arch Version Repo Size 2024-09-10T18:52:18.416 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-10T18:52:18.416 INFO:teuthology.orchestra.run.smithi047.stdout:Installing: 2024-09-10T18:52:18.416 INFO:teuthology.orchestra.run.smithi047.stdout: librados-devel x86_64 2:17.2.7-1606.g5be57395.el9 ceph 126 k 2024-09-10T18:52:18.416 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:52:18.416 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction Summary 2024-09-10T18:52:18.416 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-10T18:52:18.416 INFO:teuthology.orchestra.run.smithi047.stdout:Install 1 Package 2024-09-10T18:52:18.416 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:52:18.417 INFO:teuthology.orchestra.run.smithi047.stdout:Total download size: 126 k 2024-09-10T18:52:18.417 INFO:teuthology.orchestra.run.smithi047.stdout:Installed size: 448 k 2024-09-10T18:52:18.417 INFO:teuthology.orchestra.run.smithi047.stdout:Downloading Packages: 2024-09-10T18:52:18.630 INFO:teuthology.orchestra.run.smithi047.stdout:librados-devel-17.2.7-1606.g5be57395.el9.x86_64 594 kB/s | 126 kB 00:00 2024-09-10T18:52:18.630 INFO:teuthology.orchestra.run.smithi047.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:52:18.630 INFO:teuthology.orchestra.run.smithi047.stdout:Total 589 kB/s | 126 kB 00:00 2024-09-10T18:52:18.631 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction check 2024-09-10T18:52:18.636 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction check succeeded. 2024-09-10T18:52:18.636 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction test 2024-09-10T18:52:18.704 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction test succeeded. 2024-09-10T18:52:18.704 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction 2024-09-10T18:52:18.753 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:01:30 ago on Tue 10 Sep 2024 06:50:48 PM UTC. 2024-09-10T18:52:18.865 INFO:teuthology.orchestra.run.smithi047.stdout: Preparing : 1/1 2024-09-10T18:52:18.887 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-09-10T18:52:18.888 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-09-10T18:52:18.888 INFO:teuthology.orchestra.run.smithi149.stdout: Package Arch Version Repo Size 2024-09-10T18:52:18.888 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-09-10T18:52:18.888 INFO:teuthology.orchestra.run.smithi149.stdout:Installing: 2024-09-10T18:52:18.888 INFO:teuthology.orchestra.run.smithi149.stdout: libcephfs-devel x86_64 2:17.2.7-1606.g5be57395.el9 ceph 25 k 2024-09-10T18:52:18.888 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T18:52:18.888 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction Summary 2024-09-10T18:52:18.888 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-09-10T18:52:18.888 INFO:teuthology.orchestra.run.smithi149.stdout:Install 1 Package 2024-09-10T18:52:18.888 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T18:52:18.888 INFO:teuthology.orchestra.run.smithi149.stdout:Total download size: 25 k 2024-09-10T18:52:18.888 INFO:teuthology.orchestra.run.smithi149.stdout:Installed size: 110 k 2024-09-10T18:52:18.889 INFO:teuthology.orchestra.run.smithi149.stdout:Downloading Packages: 2024-09-10T18:52:18.971 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : librados-devel-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:52:19.041 INFO:teuthology.orchestra.run.smithi149.stdout:libcephfs-devel-17.2.7-1606.g5be57395.el9.x86_6 163 kB/s | 25 kB 00:00 2024-09-10T18:52:19.042 INFO:teuthology.orchestra.run.smithi149.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:52:19.042 INFO:teuthology.orchestra.run.smithi149.stdout:Total 161 kB/s | 25 kB 00:00 2024-09-10T18:52:19.043 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction check 2024-09-10T18:52:19.046 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction check succeeded. 2024-09-10T18:52:19.046 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction test 2024-09-10T18:52:19.047 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction test succeeded. 2024-09-10T18:52:19.048 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction 2024-09-10T18:52:19.128 INFO:teuthology.orchestra.run.smithi149.stdout: Preparing : 1/1 2024-09-10T18:52:19.229 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libcephfs-devel-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:52:19.465 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: librados-devel-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:52:19.640 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:52:19.720 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librados-devel-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:52:19.720 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:52:19.720 INFO:teuthology.orchestra.run.smithi047.stdout:Installed: 2024-09-10T18:52:19.720 INFO:teuthology.orchestra.run.smithi047.stdout: librados-devel-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:52:19.720 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:52:19.720 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-09-10T18:52:19.877 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libcephfs-devel-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:52:19.877 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T18:52:19.878 INFO:teuthology.orchestra.run.smithi149.stdout:Installed: 2024-09-10T18:52:19.878 INFO:teuthology.orchestra.run.smithi149.stdout: libcephfs-devel-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:52:19.878 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T18:52:19.878 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-09-10T18:52:20.027 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install libcephfs2 2024-09-10T18:52:20.063 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install librados2 2024-09-10T18:52:20.624 INFO:teuthology.orchestra.run.smithi047.stdout:Last metadata expiration check: 0:01:33 ago on Tue 10 Sep 2024 06:50:47 PM UTC. 2024-09-10T18:52:20.634 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:01:32 ago on Tue 10 Sep 2024 06:50:48 PM UTC. 2024-09-10T18:52:20.718 INFO:teuthology.orchestra.run.smithi047.stdout:Package libcephfs2-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T18:52:20.723 INFO:teuthology.orchestra.run.smithi149.stdout:Package librados2-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T18:52:20.764 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-09-10T18:52:20.765 INFO:teuthology.orchestra.run.smithi047.stdout:Nothing to do. 2024-09-10T18:52:20.765 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-09-10T18:52:20.766 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-09-10T18:52:20.767 INFO:teuthology.orchestra.run.smithi149.stdout:Nothing to do. 2024-09-10T18:52:20.767 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-09-10T18:52:20.821 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install libcephfs-devel 2024-09-10T18:52:20.851 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install librbd1 2024-09-10T18:52:21.412 INFO:teuthology.orchestra.run.smithi047.stdout:Last metadata expiration check: 0:01:34 ago on Tue 10 Sep 2024 06:50:47 PM UTC. 2024-09-10T18:52:21.430 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:01:33 ago on Tue 10 Sep 2024 06:50:48 PM UTC. 2024-09-10T18:52:21.521 INFO:teuthology.orchestra.run.smithi149.stdout:Package librbd1-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T18:52:21.551 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-09-10T18:52:21.552 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-10T18:52:21.552 INFO:teuthology.orchestra.run.smithi047.stdout: Package Arch Version Repo Size 2024-09-10T18:52:21.552 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-10T18:52:21.552 INFO:teuthology.orchestra.run.smithi047.stdout:Installing: 2024-09-10T18:52:21.552 INFO:teuthology.orchestra.run.smithi047.stdout: libcephfs-devel x86_64 2:17.2.7-1606.g5be57395.el9 ceph 25 k 2024-09-10T18:52:21.552 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:52:21.552 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction Summary 2024-09-10T18:52:21.552 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-10T18:52:21.553 INFO:teuthology.orchestra.run.smithi047.stdout:Install 1 Package 2024-09-10T18:52:21.553 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:52:21.553 INFO:teuthology.orchestra.run.smithi047.stdout:Total download size: 25 k 2024-09-10T18:52:21.553 INFO:teuthology.orchestra.run.smithi047.stdout:Installed size: 110 k 2024-09-10T18:52:21.553 INFO:teuthology.orchestra.run.smithi047.stdout:Downloading Packages: 2024-09-10T18:52:21.563 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-09-10T18:52:21.564 INFO:teuthology.orchestra.run.smithi149.stdout:Nothing to do. 2024-09-10T18:52:21.565 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-09-10T18:52:21.639 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install python3-rados 2024-09-10T18:52:21.709 INFO:teuthology.orchestra.run.smithi047.stdout:libcephfs-devel-17.2.7-1606.g5be57395.el9.x86_6 160 kB/s | 25 kB 00:00 2024-09-10T18:52:21.710 INFO:teuthology.orchestra.run.smithi047.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:52:21.710 INFO:teuthology.orchestra.run.smithi047.stdout:Total 159 kB/s | 25 kB 00:00 2024-09-10T18:52:21.710 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction check 2024-09-10T18:52:21.713 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction check succeeded. 2024-09-10T18:52:21.713 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction test 2024-09-10T18:52:21.715 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction test succeeded. 2024-09-10T18:52:21.715 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction 2024-09-10T18:52:21.802 INFO:teuthology.orchestra.run.smithi047.stdout: Preparing : 1/1 2024-09-10T18:52:21.917 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libcephfs-devel-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:52:22.214 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:01:34 ago on Tue 10 Sep 2024 06:50:48 PM UTC. 2024-09-10T18:52:22.304 INFO:teuthology.orchestra.run.smithi149.stdout:Package python3-rados-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T18:52:22.347 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-09-10T18:52:22.348 INFO:teuthology.orchestra.run.smithi149.stdout:Nothing to do. 2024-09-10T18:52:22.348 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-09-10T18:52:22.415 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:52:22.432 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install python3-rgw 2024-09-10T18:52:22.645 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libcephfs-devel-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:52:22.646 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:52:22.646 INFO:teuthology.orchestra.run.smithi047.stdout:Installed: 2024-09-10T18:52:22.646 INFO:teuthology.orchestra.run.smithi047.stdout: libcephfs-devel-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:52:22.646 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:52:22.646 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-09-10T18:52:22.843 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install librados2 2024-09-10T18:52:23.001 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:01:34 ago on Tue 10 Sep 2024 06:50:48 PM UTC. 2024-09-10T18:52:23.092 INFO:teuthology.orchestra.run.smithi149.stdout:Package python3-rgw-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T18:52:23.135 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-09-10T18:52:23.136 INFO:teuthology.orchestra.run.smithi149.stdout:Nothing to do. 2024-09-10T18:52:23.136 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-09-10T18:52:23.215 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install python3-cephfs 2024-09-10T18:52:23.428 INFO:teuthology.orchestra.run.smithi047.stdout:Last metadata expiration check: 0:01:36 ago on Tue 10 Sep 2024 06:50:47 PM UTC. 2024-09-10T18:52:23.519 INFO:teuthology.orchestra.run.smithi047.stdout:Package librados2-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T18:52:23.564 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-09-10T18:52:23.565 INFO:teuthology.orchestra.run.smithi047.stdout:Nothing to do. 2024-09-10T18:52:23.565 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-09-10T18:52:23.617 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install librbd1 2024-09-10T18:52:23.786 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:01:35 ago on Tue 10 Sep 2024 06:50:48 PM UTC. 2024-09-10T18:52:23.878 INFO:teuthology.orchestra.run.smithi149.stdout:Package python3-cephfs-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T18:52:23.922 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-09-10T18:52:23.923 INFO:teuthology.orchestra.run.smithi149.stdout:Nothing to do. 2024-09-10T18:52:23.924 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-09-10T18:52:24.008 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install python3-rbd 2024-09-10T18:52:24.203 INFO:teuthology.orchestra.run.smithi047.stdout:Last metadata expiration check: 0:01:37 ago on Tue 10 Sep 2024 06:50:47 PM UTC. 2024-09-10T18:52:24.295 INFO:teuthology.orchestra.run.smithi047.stdout:Package librbd1-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T18:52:24.341 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-09-10T18:52:24.342 INFO:teuthology.orchestra.run.smithi047.stdout:Nothing to do. 2024-09-10T18:52:24.342 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-09-10T18:52:24.403 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install python3-rados 2024-09-10T18:52:24.636 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:01:36 ago on Tue 10 Sep 2024 06:50:48 PM UTC. 2024-09-10T18:52:24.727 INFO:teuthology.orchestra.run.smithi149.stdout:Package python3-rbd-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T18:52:24.770 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-09-10T18:52:24.771 INFO:teuthology.orchestra.run.smithi149.stdout:Nothing to do. 2024-09-10T18:52:24.771 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-09-10T18:52:24.850 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install rbd-fuse 2024-09-10T18:52:24.987 INFO:teuthology.orchestra.run.smithi047.stdout:Last metadata expiration check: 0:01:37 ago on Tue 10 Sep 2024 06:50:47 PM UTC. 2024-09-10T18:52:25.081 INFO:teuthology.orchestra.run.smithi047.stdout:Package python3-rados-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T18:52:25.127 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-09-10T18:52:25.128 INFO:teuthology.orchestra.run.smithi047.stdout:Nothing to do. 2024-09-10T18:52:25.128 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-09-10T18:52:25.197 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install python3-rgw 2024-09-10T18:52:25.425 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:01:37 ago on Tue 10 Sep 2024 06:50:48 PM UTC. 2024-09-10T18:52:25.559 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-09-10T18:52:25.560 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-09-10T18:52:25.560 INFO:teuthology.orchestra.run.smithi149.stdout: Package Architecture Version Repository Size 2024-09-10T18:52:25.560 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-09-10T18:52:25.560 INFO:teuthology.orchestra.run.smithi149.stdout:Installing: 2024-09-10T18:52:25.561 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-fuse x86_64 2:17.2.7-1606.g5be57395.el9 ceph 82 k 2024-09-10T18:52:25.561 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T18:52:25.561 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction Summary 2024-09-10T18:52:25.561 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-09-10T18:52:25.561 INFO:teuthology.orchestra.run.smithi149.stdout:Install 1 Package 2024-09-10T18:52:25.561 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T18:52:25.562 INFO:teuthology.orchestra.run.smithi149.stdout:Total download size: 82 k 2024-09-10T18:52:25.562 INFO:teuthology.orchestra.run.smithi149.stdout:Installed size: 222 k 2024-09-10T18:52:25.562 INFO:teuthology.orchestra.run.smithi149.stdout:Downloading Packages: 2024-09-10T18:52:25.736 INFO:teuthology.orchestra.run.smithi149.stdout:rbd-fuse-17.2.7-1606.g5be57395.el9.x86_64.rpm 465 kB/s | 82 kB 00:00 2024-09-10T18:52:25.737 INFO:teuthology.orchestra.run.smithi149.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:52:25.737 INFO:teuthology.orchestra.run.smithi149.stdout:Total 462 kB/s | 82 kB 00:00 2024-09-10T18:52:25.737 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction check 2024-09-10T18:52:25.744 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction check succeeded. 2024-09-10T18:52:25.744 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction test 2024-09-10T18:52:25.790 INFO:teuthology.orchestra.run.smithi047.stdout:Last metadata expiration check: 0:01:38 ago on Tue 10 Sep 2024 06:50:47 PM UTC. 2024-09-10T18:52:25.807 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction test succeeded. 2024-09-10T18:52:25.807 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction 2024-09-10T18:52:25.883 INFO:teuthology.orchestra.run.smithi047.stdout:Package python3-rgw-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T18:52:25.929 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-09-10T18:52:25.930 INFO:teuthology.orchestra.run.smithi047.stdout:Nothing to do. 2024-09-10T18:52:25.930 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-09-10T18:52:25.951 INFO:teuthology.orchestra.run.smithi149.stdout: Preparing : 1/1 2024-09-10T18:52:25.994 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install python3-cephfs 2024-09-10T18:52:26.050 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : rbd-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:52:26.414 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: rbd-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:52:26.583 INFO:teuthology.orchestra.run.smithi047.stdout:Last metadata expiration check: 0:01:39 ago on Tue 10 Sep 2024 06:50:47 PM UTC. 2024-09-10T18:52:26.678 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : rbd-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:52:26.678 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T18:52:26.678 INFO:teuthology.orchestra.run.smithi149.stdout:Installed: 2024-09-10T18:52:26.678 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:52:26.678 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T18:52:26.678 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-09-10T18:52:26.679 INFO:teuthology.orchestra.run.smithi047.stdout:Package python3-cephfs-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T18:52:26.724 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-09-10T18:52:26.725 INFO:teuthology.orchestra.run.smithi047.stdout:Nothing to do. 2024-09-10T18:52:26.725 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-09-10T18:52:26.787 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install python3-rbd 2024-09-10T18:52:26.938 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install rbd-mirror 2024-09-10T18:52:27.380 INFO:teuthology.orchestra.run.smithi047.stdout:Last metadata expiration check: 0:01:40 ago on Tue 10 Sep 2024 06:50:47 PM UTC. 2024-09-10T18:52:27.473 INFO:teuthology.orchestra.run.smithi047.stdout:Package python3-rbd-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T18:52:27.518 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-09-10T18:52:27.519 INFO:teuthology.orchestra.run.smithi047.stdout:Nothing to do. 2024-09-10T18:52:27.520 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-09-10T18:52:27.528 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:01:39 ago on Tue 10 Sep 2024 06:50:48 PM UTC. 2024-09-10T18:52:27.587 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install rbd-fuse 2024-09-10T18:52:27.667 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-09-10T18:52:27.667 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-09-10T18:52:27.668 INFO:teuthology.orchestra.run.smithi149.stdout: Package Arch Version Repo Size 2024-09-10T18:52:27.668 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-09-10T18:52:27.668 INFO:teuthology.orchestra.run.smithi149.stdout:Installing: 2024-09-10T18:52:27.668 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-mirror x86_64 2:17.2.7-1606.g5be57395.el9 ceph 2.9 M 2024-09-10T18:52:27.668 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T18:52:27.668 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction Summary 2024-09-10T18:52:27.668 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-09-10T18:52:27.668 INFO:teuthology.orchestra.run.smithi149.stdout:Install 1 Package 2024-09-10T18:52:27.668 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T18:52:27.669 INFO:teuthology.orchestra.run.smithi149.stdout:Total download size: 2.9 M 2024-09-10T18:52:27.669 INFO:teuthology.orchestra.run.smithi149.stdout:Installed size: 12 M 2024-09-10T18:52:27.669 INFO:teuthology.orchestra.run.smithi149.stdout:Downloading Packages: 2024-09-10T18:52:28.007 INFO:teuthology.orchestra.run.smithi149.stdout:rbd-mirror-17.2.7-1606.g5be57395.el9.x86_64.rpm 8.7 MB/s | 2.9 MB 00:00 2024-09-10T18:52:28.007 INFO:teuthology.orchestra.run.smithi149.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:52:28.008 INFO:teuthology.orchestra.run.smithi149.stdout:Total 8.7 MB/s | 2.9 MB 00:00 2024-09-10T18:52:28.008 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction check 2024-09-10T18:52:28.017 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction check succeeded. 2024-09-10T18:52:28.017 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction test 2024-09-10T18:52:28.098 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction test succeeded. 2024-09-10T18:52:28.098 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction 2024-09-10T18:52:28.178 INFO:teuthology.orchestra.run.smithi047.stdout:Last metadata expiration check: 0:01:41 ago on Tue 10 Sep 2024 06:50:47 PM UTC. 2024-09-10T18:52:28.315 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-09-10T18:52:28.316 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-10T18:52:28.317 INFO:teuthology.orchestra.run.smithi047.stdout: Package Architecture Version Repository Size 2024-09-10T18:52:28.317 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-10T18:52:28.317 INFO:teuthology.orchestra.run.smithi047.stdout:Installing: 2024-09-10T18:52:28.317 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-fuse x86_64 2:17.2.7-1606.g5be57395.el9 ceph 82 k 2024-09-10T18:52:28.317 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:52:28.317 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction Summary 2024-09-10T18:52:28.317 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-10T18:52:28.317 INFO:teuthology.orchestra.run.smithi047.stdout:Install 1 Package 2024-09-10T18:52:28.317 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:52:28.317 INFO:teuthology.orchestra.run.smithi047.stdout:Total download size: 82 k 2024-09-10T18:52:28.317 INFO:teuthology.orchestra.run.smithi047.stdout:Installed size: 222 k 2024-09-10T18:52:28.317 INFO:teuthology.orchestra.run.smithi047.stdout:Downloading Packages: 2024-09-10T18:52:28.504 INFO:teuthology.orchestra.run.smithi047.stdout:rbd-fuse-17.2.7-1606.g5be57395.el9.x86_64.rpm 437 kB/s | 82 kB 00:00 2024-09-10T18:52:28.505 INFO:teuthology.orchestra.run.smithi047.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:52:28.506 INFO:teuthology.orchestra.run.smithi047.stdout:Total 434 kB/s | 82 kB 00:00 2024-09-10T18:52:28.506 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction check 2024-09-10T18:52:28.512 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction check succeeded. 2024-09-10T18:52:28.512 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction test 2024-09-10T18:52:28.519 INFO:teuthology.orchestra.run.smithi149.stdout: Preparing : 1/1 2024-09-10T18:52:28.579 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction test succeeded. 2024-09-10T18:52:28.579 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction 2024-09-10T18:52:28.591 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : rbd-mirror-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:52:28.615 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: rbd-mirror-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:52:28.615 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T18:52:28.615 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-10T18:52:28.615 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T18:52:28.616 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T18:52:28.616 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T18:52:28.728 INFO:teuthology.orchestra.run.smithi047.stdout: Preparing : 1/1 2024-09-10T18:52:28.838 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : rbd-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:52:29.227 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: rbd-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:52:29.567 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : rbd-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:52:29.567 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:52:29.567 INFO:teuthology.orchestra.run.smithi047.stdout:Installed: 2024-09-10T18:52:29.567 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:52:29.568 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:52:29.568 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-09-10T18:52:29.674 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : rbd-mirror-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:52:29.675 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T18:52:29.675 INFO:teuthology.orchestra.run.smithi149.stdout:Installed: 2024-09-10T18:52:29.675 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-mirror-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:52:29.675 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T18:52:29.675 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-09-10T18:52:29.800 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install rbd-mirror 2024-09-10T18:52:29.922 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install rbd-nbd 2024-09-10T18:52:30.404 INFO:teuthology.orchestra.run.smithi047.stdout:Last metadata expiration check: 0:01:43 ago on Tue 10 Sep 2024 06:50:47 PM UTC. 2024-09-10T18:52:30.510 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:01:42 ago on Tue 10 Sep 2024 06:50:48 PM UTC. 2024-09-10T18:52:30.544 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-09-10T18:52:30.545 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-10T18:52:30.545 INFO:teuthology.orchestra.run.smithi047.stdout: Package Arch Version Repo Size 2024-09-10T18:52:30.545 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-10T18:52:30.546 INFO:teuthology.orchestra.run.smithi047.stdout:Installing: 2024-09-10T18:52:30.546 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-mirror x86_64 2:17.2.7-1606.g5be57395.el9 ceph 2.9 M 2024-09-10T18:52:30.546 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:52:30.546 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction Summary 2024-09-10T18:52:30.546 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-10T18:52:30.546 INFO:teuthology.orchestra.run.smithi047.stdout:Install 1 Package 2024-09-10T18:52:30.546 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:52:30.546 INFO:teuthology.orchestra.run.smithi047.stdout:Total download size: 2.9 M 2024-09-10T18:52:30.546 INFO:teuthology.orchestra.run.smithi047.stdout:Installed size: 12 M 2024-09-10T18:52:30.546 INFO:teuthology.orchestra.run.smithi047.stdout:Downloading Packages: 2024-09-10T18:52:30.643 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-09-10T18:52:30.644 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-09-10T18:52:30.644 INFO:teuthology.orchestra.run.smithi149.stdout: Package Architecture Version Repository Size 2024-09-10T18:52:30.644 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-09-10T18:52:30.644 INFO:teuthology.orchestra.run.smithi149.stdout:Installing: 2024-09-10T18:52:30.644 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-nbd x86_64 2:17.2.7-1606.g5be57395.el9 ceph 165 k 2024-09-10T18:52:30.644 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T18:52:30.645 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction Summary 2024-09-10T18:52:30.645 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-09-10T18:52:30.645 INFO:teuthology.orchestra.run.smithi149.stdout:Install 1 Package 2024-09-10T18:52:30.645 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T18:52:30.645 INFO:teuthology.orchestra.run.smithi149.stdout:Total download size: 165 k 2024-09-10T18:52:30.645 INFO:teuthology.orchestra.run.smithi149.stdout:Installed size: 477 k 2024-09-10T18:52:30.645 INFO:teuthology.orchestra.run.smithi149.stdout:Downloading Packages: 2024-09-10T18:52:30.860 INFO:teuthology.orchestra.run.smithi149.stdout:rbd-nbd-17.2.7-1606.g5be57395.el9.x86_64.rpm 766 kB/s | 165 kB 00:00 2024-09-10T18:52:30.861 INFO:teuthology.orchestra.run.smithi149.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:52:30.861 INFO:teuthology.orchestra.run.smithi149.stdout:Total 762 kB/s | 165 kB 00:00 2024-09-10T18:52:30.861 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction check 2024-09-10T18:52:30.869 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction check succeeded. 2024-09-10T18:52:30.869 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction test 2024-09-10T18:52:30.875 INFO:teuthology.orchestra.run.smithi047.stdout:rbd-mirror-17.2.7-1606.g5be57395.el9.x86_64.rpm 8.9 MB/s | 2.9 MB 00:00 2024-09-10T18:52:30.876 INFO:teuthology.orchestra.run.smithi047.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:52:30.876 INFO:teuthology.orchestra.run.smithi047.stdout:Total 8.9 MB/s | 2.9 MB 00:00 2024-09-10T18:52:30.876 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction check 2024-09-10T18:52:30.887 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction check succeeded. 2024-09-10T18:52:30.887 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction test 2024-09-10T18:52:30.931 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction test succeeded. 2024-09-10T18:52:30.932 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction 2024-09-10T18:52:30.972 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction test succeeded. 2024-09-10T18:52:30.973 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction 2024-09-10T18:52:31.102 INFO:teuthology.orchestra.run.smithi149.stdout: Preparing : 1/1 2024-09-10T18:52:31.207 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : rbd-nbd-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:52:31.414 INFO:teuthology.orchestra.run.smithi047.stdout: Preparing : 1/1 2024-09-10T18:52:31.484 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : rbd-mirror-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:52:31.512 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: rbd-mirror-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:52:31.512 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T18:52:31.512 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-10T18:52:31.512 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T18:52:31.512 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T18:52:31.512 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:52:31.660 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: rbd-nbd-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:52:31.963 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : rbd-nbd-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:52:31.963 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T18:52:31.963 INFO:teuthology.orchestra.run.smithi149.stdout:Installed: 2024-09-10T18:52:31.964 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-nbd-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:52:31.964 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T18:52:31.964 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-09-10T18:52:32.193 DEBUG:teuthology.parallel:result is None 2024-09-10T18:52:32.644 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : rbd-mirror-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:52:32.645 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:52:32.645 INFO:teuthology.orchestra.run.smithi047.stdout:Installed: 2024-09-10T18:52:32.645 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-mirror-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:52:32.645 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:52:32.645 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-09-10T18:52:32.901 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install rbd-nbd 2024-09-10T18:52:33.498 INFO:teuthology.orchestra.run.smithi047.stdout:Last metadata expiration check: 0:01:46 ago on Tue 10 Sep 2024 06:50:47 PM UTC. 2024-09-10T18:52:33.636 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-09-10T18:52:33.637 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-10T18:52:33.637 INFO:teuthology.orchestra.run.smithi047.stdout: Package Architecture Version Repository Size 2024-09-10T18:52:33.637 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-10T18:52:33.637 INFO:teuthology.orchestra.run.smithi047.stdout:Installing: 2024-09-10T18:52:33.637 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-nbd x86_64 2:17.2.7-1606.g5be57395.el9 ceph 165 k 2024-09-10T18:52:33.637 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:52:33.637 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction Summary 2024-09-10T18:52:33.637 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-10T18:52:33.637 INFO:teuthology.orchestra.run.smithi047.stdout:Install 1 Package 2024-09-10T18:52:33.637 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:52:33.638 INFO:teuthology.orchestra.run.smithi047.stdout:Total download size: 165 k 2024-09-10T18:52:33.638 INFO:teuthology.orchestra.run.smithi047.stdout:Installed size: 477 k 2024-09-10T18:52:33.638 INFO:teuthology.orchestra.run.smithi047.stdout:Downloading Packages: 2024-09-10T18:52:33.837 INFO:teuthology.orchestra.run.smithi047.stdout:rbd-nbd-17.2.7-1606.g5be57395.el9.x86_64.rpm 829 kB/s | 165 kB 00:00 2024-09-10T18:52:33.838 INFO:teuthology.orchestra.run.smithi047.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:52:33.838 INFO:teuthology.orchestra.run.smithi047.stdout:Total 822 kB/s | 165 kB 00:00 2024-09-10T18:52:33.838 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction check 2024-09-10T18:52:33.847 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction check succeeded. 2024-09-10T18:52:33.847 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction test 2024-09-10T18:52:33.915 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction test succeeded. 2024-09-10T18:52:33.915 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction 2024-09-10T18:52:34.079 INFO:teuthology.orchestra.run.smithi047.stdout: Preparing : 1/1 2024-09-10T18:52:34.187 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : rbd-nbd-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:52:34.602 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: rbd-nbd-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:52:34.859 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : rbd-nbd-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:52:34.860 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:52:34.860 INFO:teuthology.orchestra.run.smithi047.stdout:Installed: 2024-09-10T18:52:34.860 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-nbd-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:52:34.860 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:52:34.860 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-09-10T18:52:35.121 DEBUG:teuthology.parallel:result is None 2024-09-10T18:52:35.121 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5be57395ea235535910512ae73ba8e7ff25fe588 2024-09-10T18:52:35.263 DEBUG:teuthology.orchestra.run.smithi047:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-10T18:52:35.285 INFO:teuthology.orchestra.run.smithi047.stdout:17.2.7-1606.g5be57395.el9 2024-09-10T18:52:35.286 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-1606.g5be57395.el9 2024-09-10T18:52:35.286 INFO:teuthology.task.install:The correct ceph version 17.2.7-1606.g5be57395 is installed. 2024-09-10T18:52:35.288 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5be57395ea235535910512ae73ba8e7ff25fe588 2024-09-10T18:52:35.423 DEBUG:teuthology.orchestra.run.smithi149:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-10T18:52:35.446 INFO:teuthology.orchestra.run.smithi149.stdout:17.2.7-1606.g5be57395.el9 2024-09-10T18:52:35.447 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-1606.g5be57395.el9 2024-09-10T18:52:35.447 INFO:teuthology.task.install:The correct ceph version 17.2.7-1606.g5be57395 is installed. 2024-09-10T18:52:35.449 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-10T18:52:35.449 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-09-10T18:52:35.449 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-10T18:52:35.477 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-09-10T18:52:35.477 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-10T18:52:35.516 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-10T18:52:35.516 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-09-10T18:52:35.516 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/usr/bin/daemon-helper 2024-09-10T18:52:35.544 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-10T18:52:35.610 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-09-10T18:52:35.611 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/usr/bin/daemon-helper 2024-09-10T18:52:35.638 DEBUG:teuthology.orchestra.run.smithi149:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-10T18:52:35.704 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-10T18:52:35.705 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-09-10T18:52:35.705 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-10T18:52:35.731 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-10T18:52:35.801 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-09-10T18:52:35.801 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-10T18:52:35.828 DEBUG:teuthology.orchestra.run.smithi149:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-10T18:52:35.893 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-10T18:52:35.894 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-09-10T18:52:35.894 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/usr/bin/stdin-killer 2024-09-10T18:52:35.920 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-10T18:52:35.986 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-09-10T18:52:35.986 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/usr/bin/stdin-killer 2024-09-10T18:52:36.012 DEBUG:teuthology.orchestra.run.smithi149:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-10T18:52:36.077 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-10T18:52:36.158 INFO:tasks.cephadm:Config: {'conf': {'mgr': {'debug mgr': 20, 'debug ms': 1}, 'global': {'mon election default strategy': 3, 'ms bind msgr1': False, 'ms bind msgr2': True, 'ms type': 'async'}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'debug ms': 1, 'debug osd': 20, 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': '5be57395ea235535910512ae73ba8e7ff25fe588', 'cephadm_mode': 'root'} 2024-09-10T18:52:36.158 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 2024-09-10T18:52:36.159 INFO:tasks.cephadm:Cluster fsid is d8563d7e-6fa5-11ef-bcea-c7b262605968 2024-09-10T18:52:36.159 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-10T18:52:36.159 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.47', 'mon.c': '[v2:172.21.15.47:3301,v1:172.21.15.47:6790]', 'mon.b': '172.21.15.149'} 2024-09-10T18:52:36.159 INFO:tasks.cephadm:First mon is mon.a on smithi047 2024-09-10T18:52:36.159 INFO:tasks.cephadm:First mgr is y 2024-09-10T18:52:36.160 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-10T18:52:36.160 DEBUG:teuthology.orchestra.run.smithi047:> sudo hostname $(hostname -s) 2024-09-10T18:52:36.189 DEBUG:teuthology.orchestra.run.smithi149:> sudo hostname $(hostname -s) 2024-09-10T18:52:36.214 INFO:tasks.cephadm:Downloading cephadm (repo https://git.ceph.com/ceph-ci.git ref 5be57395ea235535910512ae73ba8e7ff25fe588)... 2024-09-10T18:52:36.214 DEBUG:teuthology.orchestra.run.smithi047:> git clone https://git.ceph.com/ceph-ci.git testrepo && cd testrepo && git show 5be57395ea235535910512ae73ba8e7ff25fe588:src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-10T18:52:36.246 INFO:teuthology.orchestra.run.smithi047.stderr:Cloning into 'testrepo'... 2024-09-10T18:53:35.583 INFO:teuthology.orchestra.run.smithi047.stderr:Updating files: 90% (11529/12697) Updating files: 91% (11555/12697) Updating files: 92% (11682/12697) Updating files: 93% (11809/12697) Updating files: 94% (11936/12697) Updating files: 95% (12063/12697) Updating files: 96% (12190/12697) Updating files: 97% (12317/12697) Updating files: 98% (12444/12697) Updating files: 99% (12571/12697) Updating files: 100% (12697/12697) Updating files: 100% (12697/12697), done. 2024-09-10T18:53:35.618 INFO:teuthology.orchestra.run.smithi047.stdout:-rw-r--r--. 1 ubuntu ubuntu 378585 Sep 10 18:53 /home/ubuntu/cephtest/cephadm 2024-09-10T18:53:35.618 DEBUG:teuthology.orchestra.run.smithi149:> git clone https://git.ceph.com/ceph-ci.git testrepo && cd testrepo && git show 5be57395ea235535910512ae73ba8e7ff25fe588:src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-10T18:53:35.638 INFO:teuthology.orchestra.run.smithi149.stderr:Cloning into 'testrepo'... 2024-09-10T18:54:33.143 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r--. 1 ubuntu ubuntu 378585 Sep 10 18:54 /home/ubuntu/cephtest/cephadm 2024-09-10T18:54:33.144 DEBUG:teuthology.orchestra.run.smithi047:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-10T18:54:33.165 DEBUG:teuthology.orchestra.run.smithi149:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-10T18:54:33.214 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 on all hosts... 2024-09-10T18:54:33.214 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 pull 2024-09-10T18:54:33.217 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 pull 2024-09-10T18:54:34.750 INFO:teuthology.orchestra.run.smithi047.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588... 2024-09-10T18:54:36.709 INFO:teuthology.orchestra.run.smithi149.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588... 2024-09-10T18:55:34.490 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-09-10T18:55:34.491 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_version": "ceph version 17.2.7-1606-g5be57395 (5be57395ea235535910512ae73ba8e7ff25fe588) quincy (stable)", 2024-09-10T18:55:34.491 INFO:teuthology.orchestra.run.smithi047.stdout: "image_id": "4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9", 2024-09-10T18:55:34.491 INFO:teuthology.orchestra.run.smithi047.stdout: "repo_digests": [ 2024-09-10T18:55:34.491 INFO:teuthology.orchestra.run.smithi047.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16" 2024-09-10T18:55:34.491 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-10T18:55:34.491 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-09-10T18:55:35.449 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-09-10T18:55:35.449 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_version": "ceph version 17.2.7-1606-g5be57395 (5be57395ea235535910512ae73ba8e7ff25fe588) quincy (stable)", 2024-09-10T18:55:35.449 INFO:teuthology.orchestra.run.smithi149.stdout: "image_id": "4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9", 2024-09-10T18:55:35.449 INFO:teuthology.orchestra.run.smithi149.stdout: "repo_digests": [ 2024-09-10T18:55:35.449 INFO:teuthology.orchestra.run.smithi149.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16" 2024-09-10T18:55:35.449 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-09-10T18:55:35.449 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-09-10T18:55:35.469 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /etc/ceph 2024-09-10T18:55:35.504 DEBUG:teuthology.orchestra.run.smithi149:> sudo mkdir -p /etc/ceph 2024-09-10T18:55:35.532 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 777 /etc/ceph 2024-09-10T18:55:35.573 DEBUG:teuthology.orchestra.run.smithi149:> sudo chmod 777 /etc/ceph 2024-09-10T18:55:35.598 INFO:tasks.cephadm:Writing seed config... 2024-09-10T18:55:35.600 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-10T18:55:35.600 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-10T18:55:35.600 INFO:tasks.cephadm: override: [global] mon election default strategy = 3 2024-09-10T18:55:35.600 INFO:tasks.cephadm: override: [global] ms bind msgr1 = False 2024-09-10T18:55:35.600 INFO:tasks.cephadm: override: [global] ms bind msgr2 = True 2024-09-10T18:55:35.600 INFO:tasks.cephadm: override: [global] ms type = async 2024-09-10T18:55:35.600 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-10T18:55:35.600 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-10T18:55:35.601 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-10T18:55:35.601 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-10T18:55:35.601 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-10T18:55:35.601 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-10T18:55:35.601 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-09-10T18:55:35.601 DEBUG:teuthology.orchestra.run.smithi047:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-10T18:55:35.631 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = d8563d7e-6fa5-11ef-bcea-c7b262605968 mon election default strategy = 3 ms bind msgr1 = False ms bind msgr2 = True ms type = async [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true debug ms = 1 debug osd = 20 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true 2024-09-10T18:55:35.631 DEBUG:teuthology.orchestra.run.smithi047:mon.a> sudo journalctl -f -n 0 -u ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968@mon.a.service 2024-09-10T18:55:35.674 DEBUG:teuthology.orchestra.run.smithi047:mgr.y> sudo journalctl -f -n 0 -u ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968@mgr.y.service 2024-09-10T18:55:35.717 INFO:tasks.cephadm:Bootstrapping... 2024-09-10T18:55:35.718 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 -v bootstrap --fsid d8563d7e-6fa5-11ef-bcea-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id y --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.47 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-10T18:55:35.926 INFO:teuthology.orchestra.run.smithi047.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:55:35.926 INFO:teuthology.orchestra.run.smithi047.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588', '-v', 'bootstrap', '--fsid', 'd8563d7e-6fa5-11ef-bcea-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'y', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.47', '--skip-admin-label'] 2024-09-10T18:55:35.950 INFO:teuthology.orchestra.run.smithi047.stdout:/bin/podman: stdout 5.2.2 2024-09-10T18:55:35.951 INFO:teuthology.orchestra.run.smithi047.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-10T18:55:35.951 INFO:teuthology.orchestra.run.smithi047.stdout:Verifying podman|docker is present... 2024-09-10T18:55:35.974 INFO:teuthology.orchestra.run.smithi047.stdout:/bin/podman: stdout 5.2.2 2024-09-10T18:55:35.974 INFO:teuthology.orchestra.run.smithi047.stdout:Verifying lvm2 is present... 2024-09-10T18:55:35.975 INFO:teuthology.orchestra.run.smithi047.stdout:Verifying time synchronization is in place... 2024-09-10T18:55:35.983 INFO:teuthology.orchestra.run.smithi047.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-10T18:55:35.983 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-10T18:55:35.989 INFO:teuthology.orchestra.run.smithi047.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-10T18:55:35.989 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stdout inactive 2024-09-10T18:55:35.996 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stdout enabled 2024-09-10T18:55:36.003 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stdout active 2024-09-10T18:55:36.003 INFO:teuthology.orchestra.run.smithi047.stdout:Unit chronyd.service is enabled and running 2024-09-10T18:55:36.004 INFO:teuthology.orchestra.run.smithi047.stdout:Repeating the final host check... 2024-09-10T18:55:36.026 INFO:teuthology.orchestra.run.smithi047.stdout:/bin/podman: stdout 5.2.2 2024-09-10T18:55:36.026 INFO:teuthology.orchestra.run.smithi047.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-10T18:55:36.026 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl is present 2024-09-10T18:55:36.026 INFO:teuthology.orchestra.run.smithi047.stdout:lvcreate is present 2024-09-10T18:55:36.033 INFO:teuthology.orchestra.run.smithi047.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-10T18:55:36.033 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-10T18:55:36.039 INFO:teuthology.orchestra.run.smithi047.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-10T18:55:36.039 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stdout inactive 2024-09-10T18:55:36.047 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stdout enabled 2024-09-10T18:55:36.054 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stdout active 2024-09-10T18:55:36.054 INFO:teuthology.orchestra.run.smithi047.stdout:Unit chronyd.service is enabled and running 2024-09-10T18:55:36.054 INFO:teuthology.orchestra.run.smithi047.stdout:Host looks OK 2024-09-10T18:55:36.054 INFO:teuthology.orchestra.run.smithi047.stdout:Cluster fsid: d8563d7e-6fa5-11ef-bcea-c7b262605968 2024-09-10T18:55:36.054 INFO:teuthology.orchestra.run.smithi047.stdout:Acquiring lock 140370793391008 on /run/cephadm/d8563d7e-6fa5-11ef-bcea-c7b262605968.lock 2024-09-10T18:55:36.054 INFO:teuthology.orchestra.run.smithi047.stdout:Lock 140370793391008 acquired on /run/cephadm/d8563d7e-6fa5-11ef-bcea-c7b262605968.lock 2024-09-10T18:55:36.055 INFO:teuthology.orchestra.run.smithi047.stdout:Verifying IP 172.21.15.47 port 3300 ... 2024-09-10T18:55:36.055 INFO:teuthology.orchestra.run.smithi047.stdout:Verifying IP 172.21.15.47 port 6789 ... 2024-09-10T18:55:36.055 INFO:teuthology.orchestra.run.smithi047.stdout:Base mon IP(s) is [172.21.15.47:3300, 172.21.15.47:6789], mon addrv is [v2:172.21.15.47:3300,v1:172.21.15.47:6789] 2024-09-10T18:55:36.058 INFO:teuthology.orchestra.run.smithi047.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.47 metric 100 2024-09-10T18:55:36.058 INFO:teuthology.orchestra.run.smithi047.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.47 metric 100 2024-09-10T18:55:36.061 INFO:teuthology.orchestra.run.smithi047.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-10T18:55:36.061 INFO:teuthology.orchestra.run.smithi047.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2024-09-10T18:55:36.061 INFO:teuthology.orchestra.run.smithi047.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1778sec hoplimit 64 pref medium 2024-09-10T18:55:36.065 INFO:teuthology.orchestra.run.smithi047.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-10T18:55:36.065 INFO:teuthology.orchestra.run.smithi047.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-10T18:55:36.065 INFO:teuthology.orchestra.run.smithi047.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-10T18:55:36.065 INFO:teuthology.orchestra.run.smithi047.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-09-10T18:55:36.066 INFO:teuthology.orchestra.run.smithi047.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:148e/64 scope link 2024-09-10T18:55:36.066 INFO:teuthology.orchestra.run.smithi047.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-10T18:55:36.066 INFO:teuthology.orchestra.run.smithi047.stdout:Mon IP `172.21.15.47` is in CIDR network `172.21.0.0/20` 2024-09-10T18:55:36.066 INFO:teuthology.orchestra.run.smithi047.stdout:Mon IP `172.21.15.47` is in CIDR network `172.21.0.0/20` 2024-09-10T18:55:36.066 INFO:teuthology.orchestra.run.smithi047.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-10T18:55:36.066 INFO:teuthology.orchestra.run.smithi047.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-10T18:55:36.066 INFO:teuthology.orchestra.run.smithi047.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588... 2024-09-10T18:55:36.514 INFO:teuthology.orchestra.run.smithi047.stdout:/bin/podman: stdout 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 2024-09-10T18:55:36.514 INFO:teuthology.orchestra.run.smithi047.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588... 2024-09-10T18:55:36.514 INFO:teuthology.orchestra.run.smithi047.stdout:/bin/podman: stderr Getting image source signatures 2024-09-10T18:55:36.514 INFO:teuthology.orchestra.run.smithi047.stdout:/bin/podman: stderr Copying blob sha256:5df13064c03763eb21a89b455ab82ba4ec435f51a3f340cae76514aaf2996d4c 2024-09-10T18:55:36.514 INFO:teuthology.orchestra.run.smithi047.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-10T18:55:36.514 INFO:teuthology.orchestra.run.smithi047.stdout:/bin/podman: stderr Copying config sha256:4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 2024-09-10T18:55:36.515 INFO:teuthology.orchestra.run.smithi047.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-10T18:55:37.478 INFO:teuthology.orchestra.run.smithi047.stdout:ceph: stdout ceph version 17.2.7-1606-g5be57395 (5be57395ea235535910512ae73ba8e7ff25fe588) quincy (stable) 2024-09-10T18:55:37.479 INFO:teuthology.orchestra.run.smithi047.stdout:Ceph version: ceph version 17.2.7-1606-g5be57395 (5be57395ea235535910512ae73ba8e7ff25fe588) quincy (stable) 2024-09-10T18:55:37.479 INFO:teuthology.orchestra.run.smithi047.stdout:Extracting ceph user uid/gid from container image... 2024-09-10T18:55:38.427 INFO:teuthology.orchestra.run.smithi047.stdout:stat: stdout 167 167 2024-09-10T18:55:38.427 INFO:teuthology.orchestra.run.smithi047.stdout:Creating initial keys... 2024-09-10T18:55:39.448 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph-authtool: stdout AQAqluBm/5ChMxAAHhyt5T7lJN8rvzYthThZag== 2024-09-10T18:55:40.446 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph-authtool: stdout AQArluBmx8B0LhAAbVC/lLkAhxTwxplXbURnHg== 2024-09-10T18:55:41.401 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph-authtool: stdout AQAsluBm0dTEKxAA8EcraitHg2mj9uyD84vn1g== 2024-09-10T18:55:41.402 INFO:teuthology.orchestra.run.smithi047.stdout:Creating initial monmap... 2024-09-10T18:55:42.299 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-10T18:55:42.299 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = octopus 2024-09-10T18:55:42.299 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to d8563d7e-6fa5-11ef-bcea-c7b262605968 2024-09-10T18:55:42.300 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-10T18:55:42.300 INFO:teuthology.orchestra.run.smithi047.stdout:monmaptool for a [v2:172.21.15.47:3300,v1:172.21.15.47:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-10T18:55:42.300 INFO:teuthology.orchestra.run.smithi047.stdout:setting min_mon_release = octopus 2024-09-10T18:55:42.300 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/monmaptool: set fsid to d8563d7e-6fa5-11ef-bcea-c7b262605968 2024-09-10T18:55:42.300 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-10T18:55:42.300 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:55:42.300 INFO:teuthology.orchestra.run.smithi047.stdout:Creating mon... 2024-09-10T18:55:43.271 INFO:teuthology.orchestra.run.smithi047.stdout:create mon.a on 2024-09-10T18:55:43.545 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-10T18:55:43.709 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-10T18:55:43.871 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968.target. 2024-09-10T18:55:43.871 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968.target. 2024-09-10T18:55:44.062 INFO:teuthology.orchestra.run.smithi047.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968@mon.a 2024-09-10T18:55:44.062 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stderr Failed to reset failed state of unit ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968@mon.a.service: Unit ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968@mon.a.service not loaded. 2024-09-10T18:55:44.251 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968.target.wants/ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968@mon.a.service → /etc/systemd/system/ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968@.service. 2024-09-10T18:55:44.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:44 smithi047 podman[28634]: 2024-09-10 18:55:44.422551954 +0000 UTC m=+0.016732175 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 2024-09-10T18:55:46.039 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:45 smithi047 podman[28634]: 2024-09-10 18:55:45.65834754 +0000 UTC m=+1.252527795 container create 1f9fd84dd6bd2f04b5006c074359f91689c6334b13f81783bd9289505557f893 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mon-a, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-10T18:55:47.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:47 smithi047 podman[28634]: 2024-09-10 18:55:47.390248639 +0000 UTC m=+2.984428859 container init 1f9fd84dd6bd2f04b5006c074359f91689c6334b13f81783bd9289505557f893 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mon-a, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, maintainer=Guillaume Abrioux ) 2024-09-10T18:55:47.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:47 smithi047 podman[28634]: 2024-09-10 18:55:47.394585737 +0000 UTC m=+2.988765948 container start 1f9fd84dd6bd2f04b5006c074359f91689c6334b13f81783bd9289505557f893 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mon-a, io.buildah.version=1.37.2, ceph=True, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-10T18:55:47.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:47 smithi047 ceph-mon[28654]: mkfs d8563d7e-6fa5-11ef-bcea-c7b262605968 2024-09-10T18:55:47.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:47 smithi047 ceph-mon[28654]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-10T18:55:48.417 INFO:teuthology.orchestra.run.smithi047.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T18:55:48.417 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stdout disabled 2024-09-10T18:55:48.427 INFO:teuthology.orchestra.run.smithi047.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T18:55:48.427 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stdout inactive 2024-09-10T18:55:48.427 INFO:teuthology.orchestra.run.smithi047.stdout:firewalld.service is not enabled 2024-09-10T18:55:48.427 INFO:teuthology.orchestra.run.smithi047.stdout:Not possible to enable service . firewalld.service is not available 2024-09-10T18:55:48.427 INFO:teuthology.orchestra.run.smithi047.stdout:Waiting for mon to start... 2024-09-10T18:55:48.427 INFO:teuthology.orchestra.run.smithi047.stdout:Waiting for mon... 2024-09-10T18:55:48.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:48 smithi047 bash[28634]: 1f9fd84dd6bd2f04b5006c074359f91689c6334b13f81783bd9289505557f893 2024-09-10T18:55:48.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:48 smithi047 systemd[1]: Started Ceph mon.a for d8563d7e-6fa5-11ef-bcea-c7b262605968. 2024-09-10T18:55:48.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:48 smithi047 ceph-mon[28654]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-10T18:55:48.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:48 smithi047 ceph-mon[28654]: monmap e1: 1 mons at {a=[v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-10T18:55:48.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:48 smithi047 ceph-mon[28654]: fsmap 2024-09-10T18:55:48.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:48 smithi047 ceph-mon[28654]: osdmap e1: 0 total, 0 up, 0 in 2024-09-10T18:55:48.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:48 smithi047 ceph-mon[28654]: mgrmap e1: no daemons active 2024-09-10T18:55:51.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:51 smithi047 ceph-mon[28654]: from='client.? 172.21.15.47:0/565542738' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-10T18:55:54.250 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout cluster: 2024-09-10T18:55:54.252 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout id: d8563d7e-6fa5-11ef-bcea-c7b262605968 2024-09-10T18:55:54.252 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-10T18:55:54.252 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 2024-09-10T18:55:54.252 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout services: 2024-09-10T18:55:54.252 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 4s) 2024-09-10T18:55:54.252 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-10T18:55:54.252 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-10T18:55:54.252 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 2024-09-10T18:55:54.252 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout data: 2024-09-10T18:55:54.252 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-10T18:55:54.253 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-10T18:55:54.253 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-10T18:55:54.253 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout pgs: 2024-09-10T18:55:54.253 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 2024-09-10T18:55:54.253 INFO:teuthology.orchestra.run.smithi047.stdout:mon is available 2024-09-10T18:55:54.253 INFO:teuthology.orchestra.run.smithi047.stdout:Assimilating anything we can from ceph.conf... 2024-09-10T18:55:54.938 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:54 smithi047 ceph-mon[28654]: from='client.? 172.21.15.47:0/3171601527' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-10T18:55:54.938 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:54 smithi047 ceph-mon[28654]: from='client.? 172.21.15.47:0/3171601527' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-10T18:55:55.290 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 2024-09-10T18:55:55.291 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout [global] 2024-09-10T18:55:55.291 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout fsid = d8563d7e-6fa5-11ef-bcea-c7b262605968 2024-09-10T18:55:55.291 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-10T18:55:55.291 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.47:3300,v1:172.21.15.47:6789] 2024-09-10T18:55:55.291 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-10T18:55:55.291 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-10T18:55:55.291 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-10T18:55:55.291 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-10T18:55:55.291 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 2024-09-10T18:55:55.291 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-10T18:55:55.291 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-10T18:55:55.291 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 2024-09-10T18:55:55.291 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout [osd] 2024-09-10T18:55:55.292 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-10T18:55:55.292 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-10T18:55:55.292 INFO:teuthology.orchestra.run.smithi047.stdout:Generating new minimal ceph.conf... 2024-09-10T18:55:56.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:55 smithi047 ceph-mon[28654]: from='client.? 172.21.15.47:0/1221504409' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:55:56.337 INFO:teuthology.orchestra.run.smithi047.stdout:Restarting the monitor... 2024-09-10T18:55:56.630 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:56 smithi047 systemd[1]: Stopping Ceph mon.a for d8563d7e-6fa5-11ef-bcea-c7b262605968... 2024-09-10T18:55:57.005 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:56 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mon-a[28650]: 2024-09-10T18:55:56.628+0000 7f945be72640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T18:55:57.005 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:56 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mon-a[28650]: 2024-09-10T18:55:56.628+0000 7f945be72640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-10T18:55:57.005 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:56 smithi047 podman[28894]: 2024-09-10 18:55:56.762197901 +0000 UTC m=+0.252255280 container died 1f9fd84dd6bd2f04b5006c074359f91689c6334b13f81783bd9289505557f893 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739) 2024-09-10T18:55:57.263 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 podman[28894]: 2024-09-10 18:55:57.103601536 +0000 UTC m=+0.593658919 container cleanup 1f9fd84dd6bd2f04b5006c074359f91689c6334b13f81783bd9289505557f893 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T18:55:57.263 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 bash[28894]: ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mon-a 2024-09-10T18:55:57.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 podman[28906]: 2024-09-10 18:55:57.262743813 +0000 UTC m=+0.493447379 container remove 1f9fd84dd6bd2f04b5006c074359f91689c6334b13f81783bd9289505557f893 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mon-a, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2) 2024-09-10T18:55:57.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 systemd[1]: ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-10T18:55:57.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 systemd[1]: Stopped Ceph mon.a for d8563d7e-6fa5-11ef-bcea-c7b262605968. 2024-09-10T18:55:57.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 systemd[1]: Starting Ceph mon.a for d8563d7e-6fa5-11ef-bcea-c7b262605968... 2024-09-10T18:55:57.946 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 podman[29000]: 2024-09-10 18:55:57.597717106 +0000 UTC m=+0.015965139 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 2024-09-10T18:55:57.946 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 podman[29000]: 2024-09-10 18:55:57.747060607 +0000 UTC m=+0.165308667 container create 555ddf4fbf7a1872c931d51a9e9f0afdee9a75652b73d7c2f33e750bb4b3a5e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mon-a, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-10T18:55:57.947 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 podman[29000]: 2024-09-10 18:55:57.915733729 +0000 UTC m=+0.333981747 container init 555ddf4fbf7a1872c931d51a9e9f0afdee9a75652b73d7c2f33e750bb4b3a5e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mon-a, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-10T18:55:57.947 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 podman[29000]: 2024-09-10 18:55:57.918888509 +0000 UTC m=+0.337136528 container start 555ddf4fbf7a1872c931d51a9e9f0afdee9a75652b73d7c2f33e750bb4b3a5e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mon-a, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, maintainer=Guillaume Abrioux ) 2024-09-10T18:55:57.947 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: set uid:gid to 167:167 (ceph:ceph) 2024-09-10T18:55:57.947 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: ceph version 17.2.7-1606-g5be57395 (5be57395ea235535910512ae73ba8e7ff25fe588) quincy (stable), process ceph-mon, pid 2 2024-09-10T18:55:57.947 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: pidfile_write: ignore empty --pid-file 2024-09-10T18:55:57.947 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: load: jerasure load: lrc 2024-09-10T18:55:57.947 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: RocksDB version: 6.15.5 2024-09-10T18:55:57.947 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-09-10T18:55:57.947 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Compile date Sep 9 2024 2024-09-10T18:55:57.947 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: DB SUMMARY 2024-09-10T18:55:57.947 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: DB Session ID: Z1IP4ATQZIVB8LVR1S6P 2024-09-10T18:55:57.948 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: CURRENT file: CURRENT 2024-09-10T18:55:57.948 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: IDENTITY file: IDENTITY 2024-09-10T18:55:57.948 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: MANIFEST file: MANIFEST-000009 size: 131 Bytes 2024-09-10T18:55:57.948 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-10T18:55:57.948 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000010.log size: 84433 ; 2024-09-10T18:55:57.948 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.error_if_exists: 0 2024-09-10T18:55:57.948 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.create_if_missing: 0 2024-09-10T18:55:57.948 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.paranoid_checks: 1 2024-09-10T18:55:57.948 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-10T18:55:57.948 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.env: 0x55e7b296f440 2024-09-10T18:55:57.948 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.fs: Posix File System 2024-09-10T18:55:57.948 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.info_log: 0x55e7b4e5eec0 2024-09-10T18:55:57.949 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.max_file_opening_threads: 16 2024-09-10T18:55:57.949 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.statistics: (nil) 2024-09-10T18:55:57.949 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.use_fsync: 0 2024-09-10T18:55:57.949 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.max_log_file_size: 0 2024-09-10T18:55:57.949 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-10T18:55:57.949 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-10T18:55:57.949 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.keep_log_file_num: 1000 2024-09-10T18:55:57.949 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.recycle_log_file_num: 0 2024-09-10T18:55:57.949 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.allow_fallocate: 1 2024-09-10T18:55:57.949 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.allow_mmap_reads: 0 2024-09-10T18:55:57.949 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.allow_mmap_writes: 0 2024-09-10T18:55:57.949 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.use_direct_reads: 0 2024-09-10T18:55:57.949 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-10T18:55:57.949 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.create_missing_column_families: 0 2024-09-10T18:55:57.949 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.db_log_dir: 2024-09-10T18:55:57.950 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-a/store.db 2024-09-10T18:55:57.950 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-10T18:55:57.950 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-10T18:55:57.950 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-10T18:55:57.950 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-10T18:55:57.950 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-10T18:55:57.950 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-10T18:55:57.950 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.advise_random_on_open: 1 2024-09-10T18:55:57.950 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.db_write_buffer_size: 0 2024-09-10T18:55:57.950 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.write_buffer_manager: 0x55e7b4f1c7e0 2024-09-10T18:55:57.950 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-10T18:55:57.950 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-09-10T18:55:57.950 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-10T18:55:57.950 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-10T18:55:57.950 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.rate_limiter: (nil) 2024-09-10T18:55:57.950 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-10T18:55:57.952 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.wal_recovery_mode: 2 2024-09-10T18:55:57.952 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.enable_thread_tracking: 0 2024-09-10T18:55:57.952 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.enable_pipelined_write: 0 2024-09-10T18:55:57.952 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.unordered_write: 0 2024-09-10T18:55:57.952 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-10T18:55:57.952 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-10T18:55:57.952 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-10T18:55:57.952 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-10T18:55:57.952 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.row_cache: None 2024-09-10T18:55:57.952 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.wal_filter: None 2024-09-10T18:55:57.953 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-10T18:55:57.953 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.allow_ingest_behind: 0 2024-09-10T18:55:57.953 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.preserve_deletes: 0 2024-09-10T18:55:57.953 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.two_write_queues: 0 2024-09-10T18:55:57.953 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.manual_wal_flush: 0 2024-09-10T18:55:57.953 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.atomic_flush: 0 2024-09-10T18:55:57.953 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-10T18:55:57.953 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-10T18:55:57.953 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-10T18:55:57.953 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.log_readahead_size: 0 2024-09-10T18:55:57.953 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-10T18:55:57.953 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.best_efforts_recovery: 0 2024-09-10T18:55:57.953 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-10T18:55:57.953 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-10T18:55:57.954 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.allow_data_in_errors: 0 2024-09-10T18:55:57.954 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.db_host_id: __hostname__ 2024-09-10T18:55:57.954 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.max_background_jobs: 2 2024-09-10T18:55:57.954 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.max_background_compactions: -1 2024-09-10T18:55:57.954 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.max_subcompactions: 1 2024-09-10T18:55:57.954 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-10T18:55:57.954 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-10T18:55:57.954 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-10T18:55:57.954 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.max_total_wal_size: 0 2024-09-10T18:55:57.954 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-10T18:55:57.954 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-10T18:55:57.954 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-10T18:55:57.954 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-10T18:55:57.954 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.max_open_files: -1 2024-09-10T18:55:57.954 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.bytes_per_sync: 0 2024-09-10T18:55:57.954 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-10T18:55:57.955 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-10T18:55:57.955 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.compaction_readahead_size: 0 2024-09-10T18:55:57.955 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.max_background_flushes: -1 2024-09-10T18:55:57.955 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Compression algorithms supported: 2024-09-10T18:55:57.955 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: kZSTD supported: 0 2024-09-10T18:55:57.955 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: kXpressCompression supported: 0 2024-09-10T18:55:57.955 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: kBZip2Compression supported: 0 2024-09-10T18:55:57.955 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-10T18:55:57.955 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: kLZ4Compression supported: 1 2024-09-10T18:55:57.955 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: kZlibCompression supported: 1 2024-09-10T18:55:57.955 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: kLZ4HCCompression supported: 1 2024-09-10T18:55:57.955 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: kSnappyCompression supported: 1 2024-09-10T18:55:57.955 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-10T18:55:57.955 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000009 2024-09-10T18:55:57.955 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2024-09-10T18:55:57.956 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-10T18:55:57.956 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.merge_operator: 2024-09-10T18:55:57.956 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.compaction_filter: None 2024-09-10T18:55:57.956 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.compaction_filter_factory: None 2024-09-10T18:55:57.956 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.sst_partitioner_factory: None 2024-09-10T18:55:57.956 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-10T18:55:57.956 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-10T18:55:57.956 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55e7b4e05a38) 2024-09-10T18:55:57.956 INFO:journalctl@ceph.mon.a.smithi047.stdout: cache_index_and_filter_blocks: 1 2024-09-10T18:55:57.956 INFO:journalctl@ceph.mon.a.smithi047.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-10T18:55:57.956 INFO:journalctl@ceph.mon.a.smithi047.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-10T18:55:57.956 INFO:journalctl@ceph.mon.a.smithi047.stdout: pin_top_level_index_and_filter: 1 2024-09-10T18:55:57.956 INFO:journalctl@ceph.mon.a.smithi047.stdout: index_type: 0 2024-09-10T18:55:57.956 INFO:journalctl@ceph.mon.a.smithi047.stdout: data_block_index_type: 0 2024-09-10T18:55:57.956 INFO:journalctl@ceph.mon.a.smithi047.stdout: index_shortening: 1 2024-09-10T18:55:57.957 INFO:journalctl@ceph.mon.a.smithi047.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-10T18:55:57.957 INFO:journalctl@ceph.mon.a.smithi047.stdout: hash_index_allow_collision: 1 2024-09-10T18:55:57.957 INFO:journalctl@ceph.mon.a.smithi047.stdout: checksum: 1 2024-09-10T18:55:57.957 INFO:journalctl@ceph.mon.a.smithi047.stdout: no_block_cache: 0 2024-09-10T18:55:57.957 INFO:journalctl@ceph.mon.a.smithi047.stdout: block_cache: 0x55e7b4e3b1f0 2024-09-10T18:55:57.957 INFO:journalctl@ceph.mon.a.smithi047.stdout: block_cache_name: BinnedLRUCache 2024-09-10T18:55:57.957 INFO:journalctl@ceph.mon.a.smithi047.stdout: block_cache_options: 2024-09-10T18:55:57.957 INFO:journalctl@ceph.mon.a.smithi047.stdout: capacity : 536870912 2024-09-10T18:55:57.957 INFO:journalctl@ceph.mon.a.smithi047.stdout: num_shard_bits : 4 2024-09-10T18:55:57.957 INFO:journalctl@ceph.mon.a.smithi047.stdout: strict_capacity_limit : 0 2024-09-10T18:55:57.957 INFO:journalctl@ceph.mon.a.smithi047.stdout: high_pri_pool_ratio: 0.000 2024-09-10T18:55:57.957 INFO:journalctl@ceph.mon.a.smithi047.stdout: block_cache_compressed: (nil) 2024-09-10T18:55:57.957 INFO:journalctl@ceph.mon.a.smithi047.stdout: persistent_cache: (nil) 2024-09-10T18:55:57.957 INFO:journalctl@ceph.mon.a.smithi047.stdout: block_size: 4096 2024-09-10T18:55:57.958 INFO:journalctl@ceph.mon.a.smithi047.stdout: block_size_deviation: 10 2024-09-10T18:55:57.958 INFO:journalctl@ceph.mon.a.smithi047.stdout: block_restart_interval: 16 2024-09-10T18:55:57.958 INFO:journalctl@ceph.mon.a.smithi047.stdout: index_block_restart_interval: 1 2024-09-10T18:55:57.958 INFO:journalctl@ceph.mon.a.smithi047.stdout: metadata_block_size: 4096 2024-09-10T18:55:57.958 INFO:journalctl@ceph.mon.a.smithi047.stdout: partition_filters: 0 2024-09-10T18:55:57.958 INFO:journalctl@ceph.mon.a.smithi047.stdout: use_delta_encoding: 1 2024-09-10T18:55:57.958 INFO:journalctl@ceph.mon.a.smithi047.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2024-09-10T18:55:57.958 INFO:journalctl@ceph.mon.a.smithi047.stdout: whole_key_filtering: 1 2024-09-10T18:55:57.958 INFO:journalctl@ceph.mon.a.smithi047.stdout: verify_compression: 0 2024-09-10T18:55:57.958 INFO:journalctl@ceph.mon.a.smithi047.stdout: read_amp_bytes_per_bit: 0 2024-09-10T18:55:57.958 INFO:journalctl@ceph.mon.a.smithi047.stdout: format_version: 4 2024-09-10T18:55:57.958 INFO:journalctl@ceph.mon.a.smithi047.stdout: enable_index_compression: 1 2024-09-10T18:55:57.958 INFO:journalctl@ceph.mon.a.smithi047.stdout: block_align: 0 2024-09-10T18:55:57.959 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.write_buffer_size: 33554432 2024-09-10T18:55:57.959 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.max_write_buffer_number: 2 2024-09-10T18:55:57.959 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.compression: NoCompression 2024-09-10T18:55:57.959 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.bottommost_compression: Disabled 2024-09-10T18:55:57.959 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.prefix_extractor: nullptr 2024-09-10T18:55:57.959 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-10T18:55:57.959 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.num_levels: 7 2024-09-10T18:55:57.959 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-10T18:55:57.959 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-10T18:55:57.959 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-10T18:55:57.959 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-10T18:55:57.959 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-10T18:55:57.959 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-10T18:55:57.959 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-10T18:55:57.960 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-10T18:55:57.960 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-10T18:55:57.960 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-10T18:55:57.960 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-10T18:55:57.960 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.compression_opts.level: 32767 2024-09-10T18:55:57.960 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.compression_opts.strategy: 0 2024-09-10T18:55:57.960 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-10T18:55:57.960 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-10T18:55:57.960 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-10T18:55:57.960 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.compression_opts.enabled: false 2024-09-10T18:55:57.960 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-10T18:55:57.960 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-10T18:55:57.960 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-10T18:55:57.960 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.target_file_size_base: 67108864 2024-09-10T18:55:57.961 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-10T18:55:57.961 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-10T18:55:57.961 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-10T18:55:57.961 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-10T18:55:57.961 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-10T18:55:57.961 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-10T18:55:57.961 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-10T18:55:57.961 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-10T18:55:57.961 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-10T18:55:57.961 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-10T18:55:57.961 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-10T18:55:57.961 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-10T18:55:57.961 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-10T18:55:57.961 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.arena_block_size: 4194304 2024-09-10T18:55:57.962 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-10T18:55:57.962 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-10T18:55:57.962 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-09-10T18:55:57.962 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.disable_auto_compactions: 0 2024-09-10T18:55:57.962 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-10T18:55:57.962 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-10T18:55:57.962 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-10T18:55:57.962 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-10T18:55:57.962 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-10T18:55:57.962 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-10T18:55:57.962 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-10T18:55:57.962 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-10T18:55:57.962 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-10T18:55:57.962 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-10T18:55:57.963 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-10T18:55:57.963 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.inplace_update_support: 0 2024-09-10T18:55:57.963 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-10T18:55:57.963 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-10T18:55:57.963 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-10T18:55:57.963 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-10T18:55:57.963 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.bloom_locality: 0 2024-09-10T18:55:57.963 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.max_successive_merges: 0 2024-09-10T18:55:57.963 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-10T18:55:57.963 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.paranoid_file_checks: 0 2024-09-10T18:55:57.963 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.force_consistency_checks: 1 2024-09-10T18:55:57.963 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.report_bg_io_stats: 0 2024-09-10T18:55:57.963 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.ttl: 2592000 2024-09-10T18:55:57.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-10T18:55:57.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.enable_blob_files: false 2024-09-10T18:55:57.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.min_blob_size: 0 2024-09-10T18:55:57.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.blob_file_size: 268435456 2024-09-10T18:55:57.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-10T18:55:57.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-10T18:55:57.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-10T18:55:57.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000009 succeeded,manifest_file_number is 9, next_file_number is 11, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-10T18:55:57.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 5 2024-09-10T18:55:57.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: [db/version_set.cc:4082] Creating manifest 13 2024-09-10T18:55:57.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725994557942831, "job": 1, "event": "recovery_started", "wal_files": [10]} 2024-09-10T18:55:57.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #10 mode 2 2024-09-10T18:55:57.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: [table/block_based/filter_policy.cc:991] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-09-10T18:55:57.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725994557943712, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 14, "file_size": 80909, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 79130, "index_size": 241, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 581, "raw_key_size": 10110, "raw_average_key_size": 47, "raw_value_size": 73337, "raw_average_value_size": 341, "num_data_blocks": 11, "num_entries": 215, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1725994557, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "41a1fa7d-112f-4b8f-ae8f-573f40c864c5", "db_session_id": "Z1IP4ATQZIVB8LVR1S6P"}} 2024-09-10T18:55:57.965 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: [db/version_set.cc:4082] Creating manifest 15 2024-09-10T18:55:57.965 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725994557944077, "job": 1, "event": "recovery_finished"} 2024-09-10T18:55:57.965 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000010.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-10T18:55:57.965 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x55e7b4e6e700 2024-09-10T18:55:57.965 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: DB pointer 0x55e7b4e3e000 2024-09-10T18:55:57.965 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2024-09-10T18:55:57.965 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: rocksdb: [db/db_impl/db_impl.cc:903] 2024-09-10T18:55:57.965 INFO:journalctl@ceph.mon.a.smithi047.stdout: ** DB Stats ** 2024-09-10T18:55:57.965 INFO:journalctl@ceph.mon.a.smithi047.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T18:55:57.965 INFO:journalctl@ceph.mon.a.smithi047.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-10T18:55:57.969 INFO:journalctl@ceph.mon.a.smithi047.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T18:55:57.969 INFO:journalctl@ceph.mon.a.smithi047.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T18:55:57.969 INFO:journalctl@ceph.mon.a.smithi047.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-10T18:55:57.969 INFO:journalctl@ceph.mon.a.smithi047.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-09-10T18:55:57.969 INFO:journalctl@ceph.mon.a.smithi047.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T18:55:57.969 INFO:journalctl@ceph.mon.a.smithi047.stdout: 2024-09-10T18:55:57.969 INFO:journalctl@ceph.mon.a.smithi047.stdout: ** Compaction Stats [default] ** 2024-09-10T18:55:57.969 INFO:journalctl@ceph.mon.a.smithi047.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T18:55:57.969 INFO:journalctl@ceph.mon.a.smithi047.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T18:55:57.969 INFO:journalctl@ceph.mon.a.smithi047.stdout: L0 2/0 80.80 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 130.8 0.00 0.00 1 0.001 0 0 2024-09-10T18:55:57.969 INFO:journalctl@ceph.mon.a.smithi047.stdout: Sum 2/0 80.80 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 130.8 0.00 0.00 1 0.001 0 0 2024-09-10T18:55:57.969 INFO:journalctl@ceph.mon.a.smithi047.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 130.8 0.00 0.00 1 0.001 0 0 2024-09-10T18:55:57.969 INFO:journalctl@ceph.mon.a.smithi047.stdout: 2024-09-10T18:55:57.969 INFO:journalctl@ceph.mon.a.smithi047.stdout: ** Compaction Stats [default] ** 2024-09-10T18:55:57.970 INFO:journalctl@ceph.mon.a.smithi047.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T18:55:57.970 INFO:journalctl@ceph.mon.a.smithi047.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T18:55:57.970 INFO:journalctl@ceph.mon.a.smithi047.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 130.8 0.00 0.00 1 0.001 0 0 2024-09-10T18:55:57.970 INFO:journalctl@ceph.mon.a.smithi047.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T18:55:57.970 INFO:journalctl@ceph.mon.a.smithi047.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-10T18:55:57.970 INFO:journalctl@ceph.mon.a.smithi047.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-10T18:55:57.970 INFO:journalctl@ceph.mon.a.smithi047.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-10T18:55:57.970 INFO:journalctl@ceph.mon.a.smithi047.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-10T18:55:57.970 INFO:journalctl@ceph.mon.a.smithi047.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-10T18:55:57.970 INFO:journalctl@ceph.mon.a.smithi047.stdout: Cumulative compaction: 0.00 GB write, 23.51 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T18:55:57.970 INFO:journalctl@ceph.mon.a.smithi047.stdout: Interval compaction: 0.00 GB write, 23.51 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T18:55:57.970 INFO:journalctl@ceph.mon.a.smithi047.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-10T18:55:57.970 INFO:journalctl@ceph.mon.a.smithi047.stdout: 2024-09-10T18:55:57.970 INFO:journalctl@ceph.mon.a.smithi047.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-10T18:55:57.970 INFO:journalctl@ceph.mon.a.smithi047.stdout: 2024-09-10T18:55:57.971 INFO:journalctl@ceph.mon.a.smithi047.stdout: ** Compaction Stats [default] ** 2024-09-10T18:55:57.971 INFO:journalctl@ceph.mon.a.smithi047.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T18:55:57.971 INFO:journalctl@ceph.mon.a.smithi047.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T18:55:57.971 INFO:journalctl@ceph.mon.a.smithi047.stdout: L0 2/0 80.80 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 130.8 0.00 0.00 1 0.001 0 0 2024-09-10T18:55:57.971 INFO:journalctl@ceph.mon.a.smithi047.stdout: Sum 2/0 80.80 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 130.8 0.00 0.00 1 0.001 0 0 2024-09-10T18:55:57.971 INFO:journalctl@ceph.mon.a.smithi047.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-09-10T18:55:57.971 INFO:journalctl@ceph.mon.a.smithi047.stdout: 2024-09-10T18:55:57.971 INFO:journalctl@ceph.mon.a.smithi047.stdout: ** Compaction Stats [default] ** 2024-09-10T18:55:57.971 INFO:journalctl@ceph.mon.a.smithi047.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T18:55:57.971 INFO:journalctl@ceph.mon.a.smithi047.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T18:55:57.971 INFO:journalctl@ceph.mon.a.smithi047.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 130.8 0.00 0.00 1 0.001 0 0 2024-09-10T18:55:57.971 INFO:journalctl@ceph.mon.a.smithi047.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T18:55:57.971 INFO:journalctl@ceph.mon.a.smithi047.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-10T18:55:57.971 INFO:journalctl@ceph.mon.a.smithi047.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-10T18:55:57.971 INFO:journalctl@ceph.mon.a.smithi047.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-10T18:55:57.972 INFO:journalctl@ceph.mon.a.smithi047.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-10T18:55:57.972 INFO:journalctl@ceph.mon.a.smithi047.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-10T18:55:57.972 INFO:journalctl@ceph.mon.a.smithi047.stdout: Cumulative compaction: 0.00 GB write, 23.27 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T18:55:57.972 INFO:journalctl@ceph.mon.a.smithi047.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T18:55:57.972 INFO:journalctl@ceph.mon.a.smithi047.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-10T18:55:57.972 INFO:journalctl@ceph.mon.a.smithi047.stdout: 2024-09-10T18:55:57.972 INFO:journalctl@ceph.mon.a.smithi047.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-10T18:55:57.972 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: starting mon.a rank 0 at public addrs [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] at bind addrs [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid d8563d7e-6fa5-11ef-bcea-c7b262605968 2024-09-10T18:55:57.972 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: mon.a@-1(???) e1 preinit fsid d8563d7e-6fa5-11ef-bcea-c7b262605968 2024-09-10T18:55:57.972 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: mon.a@-1(???).mds e1 new map 2024-09-10T18:55:57.972 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: mon.a@-1(???).mds e1 print_map 2024-09-10T18:55:57.972 INFO:journalctl@ceph.mon.a.smithi047.stdout: e1 2024-09-10T18:55:57.972 INFO:journalctl@ceph.mon.a.smithi047.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T18:55:57.972 INFO:journalctl@ceph.mon.a.smithi047.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T18:55:57.972 INFO:journalctl@ceph.mon.a.smithi047.stdout: legacy client fscid: -1 2024-09-10T18:55:57.973 INFO:journalctl@ceph.mon.a.smithi047.stdout: 2024-09-10T18:55:57.973 INFO:journalctl@ceph.mon.a.smithi047.stdout: No filesystems configured 2024-09-10T18:55:57.973 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-10T18:55:57.973 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T18:55:57.973 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T18:55:57.973 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T18:55:57.973 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 ceph-mon[29019]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-10T18:55:57.996 INFO:teuthology.orchestra.run.smithi047.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-10T18:55:58.228 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 bash[29000]: 555ddf4fbf7a1872c931d51a9e9f0afdee9a75652b73d7c2f33e750bb4b3a5e5 2024-09-10T18:55:58.229 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:57 smithi047 systemd[1]: Started Ceph mon.a for d8563d7e-6fa5-11ef-bcea-c7b262605968. 2024-09-10T18:55:58.229 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:58 smithi047 ceph-mon[29019]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-10T18:55:58.229 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:58 smithi047 ceph-mon[29019]: monmap e1: 1 mons at {a=[v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-10T18:55:58.229 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:58 smithi047 ceph-mon[29019]: fsmap 2024-09-10T18:55:58.229 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:58 smithi047 ceph-mon[29019]: osdmap e1: 0 total, 0 up, 0 in 2024-09-10T18:55:58.229 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:58 smithi047 ceph-mon[29019]: mgrmap e1: no daemons active 2024-09-10T18:55:58.992 INFO:teuthology.orchestra.run.smithi047.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-10T18:55:58.993 INFO:teuthology.orchestra.run.smithi047.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-10T18:55:58.993 INFO:teuthology.orchestra.run.smithi047.stdout:Creating mgr... 2024-09-10T18:55:58.993 INFO:teuthology.orchestra.run.smithi047.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-10T18:55:58.993 INFO:teuthology.orchestra.run.smithi047.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-10T18:55:59.179 INFO:teuthology.orchestra.run.smithi047.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968@mgr.y 2024-09-10T18:55:59.179 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stderr Failed to reset failed state of unit ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968@mgr.y.service: Unit ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968@mgr.y.service not loaded. 2024-09-10T18:55:59.410 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968.target.wants/ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968@mgr.y.service → /etc/systemd/system/ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968@.service. 2024-09-10T18:55:59.470 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:55:59 smithi047 systemd[1]: Starting Ceph mgr.y for d8563d7e-6fa5-11ef-bcea-c7b262605968... 2024-09-10T18:55:59.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:55:59 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/3721912721' entity='client.admin' 2024-09-10T18:55:59.790 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:55:59 smithi047 podman[29229]: 2024-09-10 18:55:59.544927148 +0000 UTC m=+0.016265407 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 2024-09-10T18:56:01.039 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:00 smithi047 podman[29229]: 2024-09-10 18:56:00.671423784 +0000 UTC m=+1.142762029 container create 42514f6762b3daf494541f2c64d9ad00d0adaa1657ca005d65ae41d413c6d07d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T18:56:01.457 INFO:teuthology.orchestra.run.smithi047.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T18:56:01.457 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stdout disabled 2024-09-10T18:56:01.465 INFO:teuthology.orchestra.run.smithi047.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T18:56:01.465 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stdout inactive 2024-09-10T18:56:01.465 INFO:teuthology.orchestra.run.smithi047.stdout:firewalld.service is not enabled 2024-09-10T18:56:01.466 INFO:teuthology.orchestra.run.smithi047.stdout:Not possible to enable service . firewalld.service is not available 2024-09-10T18:56:01.474 INFO:teuthology.orchestra.run.smithi047.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T18:56:01.475 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stdout disabled 2024-09-10T18:56:01.482 INFO:teuthology.orchestra.run.smithi047.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T18:56:01.483 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stdout inactive 2024-09-10T18:56:01.483 INFO:teuthology.orchestra.run.smithi047.stdout:firewalld.service is not enabled 2024-09-10T18:56:01.483 INFO:teuthology.orchestra.run.smithi047.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-10T18:56:01.483 INFO:teuthology.orchestra.run.smithi047.stdout:Waiting for mgr to start... 2024-09-10T18:56:01.483 INFO:teuthology.orchestra.run.smithi047.stdout:Waiting for mgr... 2024-09-10T18:56:01.532 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:01 smithi047 podman[29229]: 2024-09-10 18:56:01.342011272 +0000 UTC m=+1.813349517 container init 42514f6762b3daf494541f2c64d9ad00d0adaa1657ca005d65ae41d413c6d07d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739) 2024-09-10T18:56:01.532 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:01 smithi047 podman[29229]: 2024-09-10 18:56:01.34503876 +0000 UTC m=+1.816377010 container start 42514f6762b3daf494541f2c64d9ad00d0adaa1657ca005d65ae41d413c6d07d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True) 2024-09-10T18:56:01.532 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:01 smithi047 bash[29229]: 42514f6762b3daf494541f2c64d9ad00d0adaa1657ca005d65ae41d413c6d07d 2024-09-10T18:56:01.532 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:01 smithi047 systemd[1]: Started Ceph mgr.y for d8563d7e-6fa5-11ef-bcea-c7b262605968. 2024-09-10T18:56:01.532 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:01 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T18:56:01.466+0000 7f537b3a8ec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T18:56:01.532 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:01 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T18:56:01.530+0000 7f537b3a8ec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T18:56:01.789 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:01 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T18:56:01.597+0000 7f537b3a8ec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T18:56:01.789 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:01 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T18:56:01.729+0000 7f537b3a8ec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T18:56:02.152 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:02 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/4024978391' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-10T18:56:02.152 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:01 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T18:56:01.872+0000 7f537b3a8ec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T18:56:02.152 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:02 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T18:56:02.150+0000 7f537b3a8ec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T18:56:02.539 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:02 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T18:56:02.214+0000 7f537b3a8ec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T18:56:02.540 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:02 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T18:56:02.383+0000 7f537b3a8ec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T18:56:02.587 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 2024-09-10T18:56:02.587 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout { 2024-09-10T18:56:02.587 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "fsid": "d8563d7e-6fa5-11ef-bcea-c7b262605968", 2024-09-10T18:56:02.587 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "health": { 2024-09-10T18:56:02.587 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-10T18:56:02.587 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-10T18:56:02.588 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-10T18:56:02.588 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:56:02.588 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-10T18:56:02.588 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-10T18:56:02.588 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 0 2024-09-10T18:56:02.588 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout ], 2024-09-10T18:56:02.588 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-10T18:56:02.588 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "a" 2024-09-10T18:56:02.588 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout ], 2024-09-10T18:56:02.588 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "quorum_age": 4, 2024-09-10T18:56:02.588 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-10T18:56:02.588 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T18:56:02.588 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-09-10T18:56:02.588 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-10T18:56:02.588 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:56:02.589 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-10T18:56:02.589 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T18:56:02.589 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-10T18:56:02.589 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-10T18:56:02.589 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-10T18:56:02.589 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-10T18:56:02.589 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-10T18:56:02.589 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-10T18:56:02.592 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:56:02.592 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-10T18:56:02.592 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-10T18:56:02.592 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-10T18:56:02.592 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-10T18:56:02.592 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-10T18:56:02.593 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-10T18:56:02.593 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-10T18:56:02.593 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-10T18:56:02.593 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-10T18:56:02.593 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:56:02.593 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-10T18:56:02.593 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T18:56:02.593 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-10T18:56:02.593 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-10T18:56:02.593 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:56:02.593 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-10T18:56:02.593 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-10T18:56:02.593 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-10T18:56:02.593 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-10T18:56:02.594 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-10T18:56:02.594 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-10T18:56:02.594 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "restful" 2024-09-10T18:56:02.594 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout ], 2024-09-10T18:56:02.594 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T18:56:02.594 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:56:02.594 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-10T18:56:02.594 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T18:56:02.594 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "modified": "2024-09-10T18:55:47.434479+0000", 2024-09-10T18:56:02.594 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T18:56:02.594 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:56:02.594 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-10T18:56:02.594 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout } 2024-09-10T18:56:02.594 INFO:teuthology.orchestra.run.smithi047.stdout:mgr not available, waiting (1/15)... 2024-09-10T18:56:03.308 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:03 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T18:56:03.277+0000 7f537b3a8ec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T18:56:03.789 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:03 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T18:56:03.346+0000 7f537b3a8ec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T18:56:04.415 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:04 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T18:56:04.107+0000 7f537b3a8ec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T18:56:04.415 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:04 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T18:56:04.224+0000 7f537b3a8ec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T18:56:04.415 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:04 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T18:56:04.351+0000 7f537b3a8ec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T18:56:04.415 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:04 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T18:56:04.413+0000 7f537b3a8ec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T18:56:04.682 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:04 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T18:56:04.544+0000 7f537b3a8ec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T18:56:04.682 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:04 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T18:56:04.619+0000 7f537b3a8ec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T18:56:04.682 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:04 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T18:56:04.680+0000 7f537b3a8ec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T18:56:04.963 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:04 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T18:56:04.825+0000 7f537b3a8ec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T18:56:04.964 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:04 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T18:56:04.892+0000 7f537b3a8ec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T18:56:04.964 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:04 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T18:56:04.961+0000 7f537b3a8ec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T18:56:05.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:05 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/1315764365' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-10T18:56:05.290 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:05 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T18:56:05.048+0000 7f537b3a8ec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T18:56:05.290 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:05 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T18:56:05.112+0000 7f537b3a8ec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T18:56:05.290 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:05 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T18:56:05.171+0000 7f537b3a8ec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T18:56:05.633 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 2024-09-10T18:56:05.636 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout { 2024-09-10T18:56:05.636 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "fsid": "d8563d7e-6fa5-11ef-bcea-c7b262605968", 2024-09-10T18:56:05.636 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "health": { 2024-09-10T18:56:05.636 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-10T18:56:05.637 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-10T18:56:05.637 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-10T18:56:05.637 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:56:05.637 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-10T18:56:05.637 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-10T18:56:05.637 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 0 2024-09-10T18:56:05.637 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout ], 2024-09-10T18:56:05.637 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-10T18:56:05.637 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "a" 2024-09-10T18:56:05.637 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout ], 2024-09-10T18:56:05.637 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "quorum_age": 7, 2024-09-10T18:56:05.637 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-10T18:56:05.637 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T18:56:05.638 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-09-10T18:56:05.638 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-10T18:56:05.638 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:56:05.638 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-10T18:56:05.638 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T18:56:05.638 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-10T18:56:05.638 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-10T18:56:05.638 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-10T18:56:05.638 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-10T18:56:05.638 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-10T18:56:05.638 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-10T18:56:05.642 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:56:05.642 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-10T18:56:05.642 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-10T18:56:05.642 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-10T18:56:05.642 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-10T18:56:05.642 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-10T18:56:05.642 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-10T18:56:05.642 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-10T18:56:05.642 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-10T18:56:05.642 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-10T18:56:05.642 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:56:05.643 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-10T18:56:05.643 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T18:56:05.643 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-10T18:56:05.643 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-10T18:56:05.643 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:56:05.643 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-10T18:56:05.643 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-10T18:56:05.643 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-10T18:56:05.643 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-10T18:56:05.643 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-10T18:56:05.643 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-10T18:56:05.643 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "restful" 2024-09-10T18:56:05.643 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout ], 2024-09-10T18:56:05.644 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T18:56:05.644 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:56:05.644 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-10T18:56:05.644 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T18:56:05.644 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "modified": "2024-09-10T18:55:47.434479+0000", 2024-09-10T18:56:05.644 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T18:56:05.644 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:56:05.644 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-10T18:56:05.644 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout } 2024-09-10T18:56:05.644 INFO:teuthology.orchestra.run.smithi047.stdout:mgr not available, waiting (2/15)... 2024-09-10T18:56:06.289 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:05 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T18:56:05.961+0000 7f537b3a8ec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T18:56:06.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:06 smithi047 ceph-mon[29019]: Activating manager daemon y 2024-09-10T18:56:06.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:06 smithi047 ceph-mon[29019]: mgrmap e2: y(active, starting, since 0.00277326s) 2024-09-10T18:56:06.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:06 smithi047 ceph-mon[29019]: from='mgr.14100 172.21.15.47:0/1267227439' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T18:56:06.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:06 smithi047 ceph-mon[29019]: from='mgr.14100 172.21.15.47:0/1267227439' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T18:56:06.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:06 smithi047 ceph-mon[29019]: from='mgr.14100 172.21.15.47:0/1267227439' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T18:56:06.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:06 smithi047 ceph-mon[29019]: from='mgr.14100 172.21.15.47:0/1267227439' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:56:06.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:06 smithi047 ceph-mon[29019]: from='mgr.14100 172.21.15.47:0/1267227439' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T18:56:06.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:06 smithi047 ceph-mon[29019]: Manager daemon y is now available 2024-09-10T18:56:06.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:06 smithi047 ceph-mon[29019]: from='mgr.14100 172.21.15.47:0/1267227439' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-10T18:56:06.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:06 smithi047 ceph-mon[29019]: from='mgr.14100 172.21.15.47:0/1267227439' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-10T18:56:06.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:06 smithi047 ceph-mon[29019]: from='mgr.14100 172.21.15.47:0/1267227439' entity='mgr.y' 2024-09-10T18:56:06.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:06 smithi047 ceph-mon[29019]: from='mgr.14100 172.21.15.47:0/1267227439' entity='mgr.y' 2024-09-10T18:56:06.291 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:06 smithi047 ceph-mon[29019]: from='mgr.14100 172.21.15.47:0/1267227439' entity='mgr.y' 2024-09-10T18:56:08.039 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:07 smithi047 ceph-mon[29019]: mgrmap e3: y(active, since 1.00684s) 2024-09-10T18:56:08.875 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 2024-09-10T18:56:08.876 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout { 2024-09-10T18:56:08.876 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "fsid": "d8563d7e-6fa5-11ef-bcea-c7b262605968", 2024-09-10T18:56:08.876 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "health": { 2024-09-10T18:56:08.876 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-10T18:56:08.876 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-10T18:56:08.876 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-10T18:56:08.876 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:56:08.876 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-10T18:56:08.876 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-10T18:56:08.876 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 0 2024-09-10T18:56:08.876 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout ], 2024-09-10T18:56:08.876 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-10T18:56:08.877 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "a" 2024-09-10T18:56:08.877 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout ], 2024-09-10T18:56:08.877 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-09-10T18:56:08.877 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-10T18:56:08.877 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T18:56:08.877 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-09-10T18:56:08.877 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-10T18:56:08.877 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:56:08.877 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-10T18:56:08.877 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T18:56:08.877 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-10T18:56:08.877 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-10T18:56:08.877 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-10T18:56:08.878 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-10T18:56:08.878 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-10T18:56:08.878 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-10T18:56:08.881 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:56:08.881 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-10T18:56:08.881 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-10T18:56:08.881 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-10T18:56:08.881 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-10T18:56:08.881 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-10T18:56:08.881 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-10T18:56:08.882 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-10T18:56:08.882 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-10T18:56:08.882 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-10T18:56:08.882 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:56:08.882 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-10T18:56:08.882 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T18:56:08.882 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-10T18:56:08.882 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-10T18:56:08.882 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:56:08.882 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-10T18:56:08.882 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-10T18:56:08.882 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-10T18:56:08.882 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-10T18:56:08.882 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-10T18:56:08.883 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-10T18:56:08.883 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "restful" 2024-09-10T18:56:08.883 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout ], 2024-09-10T18:56:08.883 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T18:56:08.883 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:56:08.883 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-10T18:56:08.883 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T18:56:08.883 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "modified": "2024-09-10T18:55:47.434479+0000", 2024-09-10T18:56:08.883 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T18:56:08.883 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:56:08.883 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-10T18:56:08.883 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout } 2024-09-10T18:56:08.883 INFO:teuthology.orchestra.run.smithi047.stdout:mgr is available 2024-09-10T18:56:09.257 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:08 smithi047 ceph-mon[29019]: mgrmap e4: y(active, since 2s) 2024-09-10T18:56:09.257 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:08 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/3433752945' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-10T18:56:09.998 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 2024-09-10T18:56:09.998 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout [global] 2024-09-10T18:56:09.998 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout fsid = d8563d7e-6fa5-11ef-bcea-c7b262605968 2024-09-10T18:56:09.998 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-10T18:56:09.998 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-10T18:56:09.998 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-10T18:56:09.998 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-10T18:56:09.998 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-10T18:56:09.998 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 2024-09-10T18:56:09.998 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-10T18:56:09.998 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-10T18:56:09.999 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 2024-09-10T18:56:09.999 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout [osd] 2024-09-10T18:56:09.999 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-10T18:56:09.999 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-10T18:56:09.999 INFO:teuthology.orchestra.run.smithi047.stdout:Enabling cephadm module... 2024-09-10T18:56:10.289 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:09 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/2670185981' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-10T18:56:11.235 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:10 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/1095799512' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-10T18:56:11.235 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:11 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: ignoring --setuser ceph since I am not root 2024-09-10T18:56:11.235 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:11 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: ignoring --setgroup ceph since I am not root 2024-09-10T18:56:11.235 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:11 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T18:56:11.109+0000 7f6fbde59ec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T18:56:11.235 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:11 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T18:56:11.166+0000 7f6fbde59ec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T18:56:11.236 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:11 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T18:56:11.233+0000 7f6fbde59ec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T18:56:11.507 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:11 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T18:56:11.363+0000 7f6fbde59ec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T18:56:11.507 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:11 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T18:56:11.505+0000 7f6fbde59ec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T18:56:12.014 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:11 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/1095799512' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-10T18:56:12.015 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:11 smithi047 ceph-mon[29019]: mgrmap e5: y(active, since 5s) 2024-09-10T18:56:12.015 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:11 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T18:56:11.777+0000 7f6fbde59ec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T18:56:12.015 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:11 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T18:56:11.840+0000 7f6fbde59ec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T18:56:12.277 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:12 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T18:56:12.012+0000 7f6fbde59ec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T18:56:12.868 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout { 2024-09-10T18:56:12.868 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-10T18:56:12.868 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-10T18:56:12.868 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-09-10T18:56:12.869 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-10T18:56:12.869 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout } 2024-09-10T18:56:12.869 INFO:teuthology.orchestra.run.smithi047.stdout:Waiting for the mgr to restart... 2024-09-10T18:56:12.869 INFO:teuthology.orchestra.run.smithi047.stdout:Waiting for mgr epoch 5... 2024-09-10T18:56:13.116 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:12 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/3354617422' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-10T18:56:13.116 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:12 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T18:56:12.915+0000 7f6fbde59ec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T18:56:13.116 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:12 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T18:56:12.983+0000 7f6fbde59ec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T18:56:14.039 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:13 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T18:56:13.731+0000 7f6fbde59ec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T18:56:14.040 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:13 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T18:56:13.850+0000 7f6fbde59ec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T18:56:14.040 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:13 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T18:56:13.977+0000 7f6fbde59ec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T18:56:14.302 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:14 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T18:56:14.038+0000 7f6fbde59ec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T18:56:14.302 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:14 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T18:56:14.170+0000 7f6fbde59ec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T18:56:14.302 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:14 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T18:56:14.240+0000 7f6fbde59ec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T18:56:14.302 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:14 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T18:56:14.300+0000 7f6fbde59ec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T18:56:14.580 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:14 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T18:56:14.445+0000 7f6fbde59ec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T18:56:14.580 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:14 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T18:56:14.511+0000 7f6fbde59ec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T18:56:14.580 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:14 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T18:56:14.579+0000 7f6fbde59ec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T18:56:15.039 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:14 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T18:56:14.663+0000 7f6fbde59ec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T18:56:15.039 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:14 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T18:56:14.723+0000 7f6fbde59ec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T18:56:15.039 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:14 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T18:56:14.782+0000 7f6fbde59ec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T18:56:16.039 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:15 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T18:56:15.560+0000 7f6fbde59ec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T18:56:16.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:15 smithi047 ceph-mon[29019]: Active manager daemon y restarted 2024-09-10T18:56:16.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:15 smithi047 ceph-mon[29019]: Activating manager daemon y 2024-09-10T18:56:16.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:15 smithi047 ceph-mon[29019]: osdmap e2: 0 total, 0 up, 0 in 2024-09-10T18:56:16.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:15 smithi047 ceph-mon[29019]: mgrmap e6: y(active, starting, since 0.00320833s) 2024-09-10T18:56:16.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:15 smithi047 ceph-mon[29019]: from='mgr.14118 172.21.15.47:0/3692814452' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:56:16.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:15 smithi047 ceph-mon[29019]: from='mgr.14118 172.21.15.47:0/3692814452' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T18:56:16.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:15 smithi047 ceph-mon[29019]: from='mgr.14118 172.21.15.47:0/3692814452' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T18:56:16.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:15 smithi047 ceph-mon[29019]: from='mgr.14118 172.21.15.47:0/3692814452' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T18:56:16.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:15 smithi047 ceph-mon[29019]: from='mgr.14118 172.21.15.47:0/3692814452' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T18:56:16.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:15 smithi047 ceph-mon[29019]: Manager daemon y is now available 2024-09-10T18:56:16.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:15 smithi047 ceph-mon[29019]: from='mgr.14118 172.21.15.47:0/3692814452' entity='mgr.y' 2024-09-10T18:56:16.041 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:15 smithi047 ceph-mon[29019]: from='mgr.14118 172.21.15.47:0/3692814452' entity='mgr.y' 2024-09-10T18:56:16.041 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:15 smithi047 ceph-mon[29019]: from='mgr.14118 172.21.15.47:0/3692814452' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:56:16.041 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:15 smithi047 ceph-mon[29019]: from='mgr.14118 172.21.15.47:0/3692814452' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:56:16.041 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:15 smithi047 ceph-mon[29019]: from='mgr.14118 172.21.15.47:0/3692814452' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-10T18:56:16.041 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:15 smithi047 ceph-mon[29019]: from='mgr.14118 172.21.15.47:0/3692814452' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-10T18:56:17.039 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:16 smithi047 ceph-mon[29019]: Found migration_current of "None". Setting to last migration. 2024-09-10T18:56:17.039 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:16 smithi047 ceph-mon[29019]: mgrmap e7: y(active, since 1.00506s) 2024-09-10T18:56:17.282 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout { 2024-09-10T18:56:17.283 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-10T18:56:17.283 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-10T18:56:17.283 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout } 2024-09-10T18:56:17.283 INFO:teuthology.orchestra.run.smithi047.stdout:mgr epoch 5 is available 2024-09-10T18:56:17.283 INFO:teuthology.orchestra.run.smithi047.stdout:Setting orchestrator backend to cephadm... 2024-09-10T18:56:17.682 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:17 smithi047 ceph-mon[29019]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-10T18:56:17.682 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:17 smithi047 ceph-mon[29019]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-10T18:56:18.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:18 smithi047 ceph-mon[29019]: mgrmap e8: y(active, since 2s) 2024-09-10T18:56:18.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:18 smithi047 ceph-mon[29019]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:56:18.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:18 smithi047 ceph-mon[29019]: from='mgr.14118 172.21.15.47:0/3692814452' entity='mgr.y' 2024-09-10T18:56:18.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:18 smithi047 ceph-mon[29019]: from='mgr.14118 172.21.15.47:0/3692814452' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:56:19.534 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:19 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T18:56:19.464+0000 7f6f70207640 -1 log_channel(cephadm) log [ERR] : [10/Sep/2024:18:56:19] ENGINE Error in HTTPServer.serve 2024-09-10T18:56:19.535 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:19 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: Traceback (most recent call last): 2024-09-10T18:56:19.535 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:19 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T18:56:19.535 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:19 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: self._connections.run(self.expiration_interval) 2024-09-10T18:56:19.535 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:19 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T18:56:19.535 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:19 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: self._run(expiration_interval) 2024-09-10T18:56:19.535 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:19 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T18:56:19.535 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:19 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T18:56:19.535 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:19 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T18:56:19.535 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:19 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T18:56:19.535 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:19 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T18:56:19.535 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:19 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: s = self.context.wrap_socket( 2024-09-10T18:56:19.535 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:19 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T18:56:19.536 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:19 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: return self.sslsocket_class._create( 2024-09-10T18:56:19.536 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:19 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T18:56:19.536 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:19 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: self.do_handshake() 2024-09-10T18:56:19.536 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:19 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T18:56:19.536 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:19 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: self._sslobj.do_handshake() 2024-09-10T18:56:19.536 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:19 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T18:56:19.536 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:19 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T18:56:19.536 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-10T18:56:19.537 INFO:teuthology.orchestra.run.smithi047.stdout:Generating ssh key... 2024-09-10T18:56:19.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:19 smithi047 ceph-mon[29019]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:56:19.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:19 smithi047 ceph-mon[29019]: from='mgr.14118 172.21.15.47:0/3692814452' entity='mgr.y' 2024-09-10T18:56:19.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:19 smithi047 ceph-mon[29019]: from='mgr.14118 172.21.15.47:0/3692814452' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:56:20.493 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:20 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: Generating public/private rsa key pair. 2024-09-10T18:56:20.493 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:20 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: Your identification has been saved in /tmp/tmp2lli9m0l/key 2024-09-10T18:56:20.493 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:20 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: Your public key has been saved in /tmp/tmp2lli9m0l/key.pub 2024-09-10T18:56:20.493 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:20 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: The key fingerprint is: 2024-09-10T18:56:20.493 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:20 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: SHA256:Fg4Egop1ZOFOr5rV+yG60GlHQnC5ylNKyUPUmW8jIyY ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968 2024-09-10T18:56:20.493 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:20 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: The key's randomart image is: 2024-09-10T18:56:20.493 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:20 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: +---[RSA 3072]----+ 2024-09-10T18:56:20.493 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:20 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: | o+o**. | 2024-09-10T18:56:20.493 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:20 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: |. o**. | 2024-09-10T18:56:20.494 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:20 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: |o+ o+o. . | 2024-09-10T18:56:20.494 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:20 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: |E B+=.+o . | 2024-09-10T18:56:20.494 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:20 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: | = *o+o.S | 2024-09-10T18:56:20.494 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:20 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: | =. * . | 2024-09-10T18:56:20.494 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:20 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: | ..* + . | 2024-09-10T18:56:20.494 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:20 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: | * o o . | 2024-09-10T18:56:20.494 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:20 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: | o o.... | 2024-09-10T18:56:20.494 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:20 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: +----[SHA256]-----+ 2024-09-10T18:56:20.782 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:20 smithi047 ceph-mon[29019]: [10/Sep/2024:18:56:19] ENGINE Bus STARTING 2024-09-10T18:56:20.782 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:20 smithi047 ceph-mon[29019]: [10/Sep/2024:18:56:19] ENGINE Serving on https://172.21.15.47:7150 2024-09-10T18:56:20.783 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:20 smithi047 ceph-mon[29019]: [10/Sep/2024:18:56:19] ENGINE Bus STARTED 2024-09-10T18:56:20.783 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:20 smithi047 ceph-mon[29019]: [10/Sep/2024:18:56:19] ENGINE Error in HTTPServer.serve 2024-09-10T18:56:20.783 INFO:journalctl@ceph.mon.a.smithi047.stdout: Traceback (most recent call last): 2024-09-10T18:56:20.783 INFO:journalctl@ceph.mon.a.smithi047.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T18:56:20.783 INFO:journalctl@ceph.mon.a.smithi047.stdout: self._connections.run(self.expiration_interval) 2024-09-10T18:56:20.783 INFO:journalctl@ceph.mon.a.smithi047.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T18:56:20.783 INFO:journalctl@ceph.mon.a.smithi047.stdout: self._run(expiration_interval) 2024-09-10T18:56:20.783 INFO:journalctl@ceph.mon.a.smithi047.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T18:56:20.783 INFO:journalctl@ceph.mon.a.smithi047.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T18:56:20.783 INFO:journalctl@ceph.mon.a.smithi047.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T18:56:20.783 INFO:journalctl@ceph.mon.a.smithi047.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T18:56:20.783 INFO:journalctl@ceph.mon.a.smithi047.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T18:56:20.783 INFO:journalctl@ceph.mon.a.smithi047.stdout: s = self.context.wrap_socket( 2024-09-10T18:56:20.784 INFO:journalctl@ceph.mon.a.smithi047.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T18:56:20.784 INFO:journalctl@ceph.mon.a.smithi047.stdout: return self.sslsocket_class._create( 2024-09-10T18:56:20.784 INFO:journalctl@ceph.mon.a.smithi047.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T18:56:20.784 INFO:journalctl@ceph.mon.a.smithi047.stdout: self.do_handshake() 2024-09-10T18:56:20.784 INFO:journalctl@ceph.mon.a.smithi047.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T18:56:20.784 INFO:journalctl@ceph.mon.a.smithi047.stdout: self._sslobj.do_handshake() 2024-09-10T18:56:20.784 INFO:journalctl@ceph.mon.a.smithi047.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T18:56:20.784 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:20 smithi047 ceph-mon[29019]: from='mgr.14118 172.21.15.47:0/3692814452' entity='mgr.y' 2024-09-10T18:56:20.784 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:20 smithi047 ceph-mon[29019]: from='mgr.14118 172.21.15.47:0/3692814452' entity='mgr.y' 2024-09-10T18:56:21.705 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:21 smithi047 ceph-mon[29019]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:56:21.705 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:21 smithi047 ceph-mon[29019]: Generating ssh key... 2024-09-10T18:56:21.966 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3paN5nDBMnsOFQW0vWC8PCBBktAtY5qw4QHN0ESfwUgnocj5QmpzSWKEVisD2cgyBM5pv7+AVJNYErwPdPYlss0QqJBJkr1vhoP62BN0VJWdxwF4ZtESE8HTLatkevft+HmA8FaIltnxtJEYA6wHGZFppaiW/p2zMwnCbxchJA8x2UW9G6hA78PBgdaj3hFV7Fv6pH/fPrWL7XoofN4rpStgGL8Z5qA3UXnzQJwxaG7ZktO3wNGjaVAtJ5aQEBJqYS4N4crMSB2qLoYOnS2X2FY7TKGAgVIT0ejBhpHeZzgVYREEp5D+2pXy94+Y55LkrQRuK26zrimTEq4+hKtu5lc9l39HWccYxajXvqrcsbhQ0JKu2XLNq9vUdbMN6TBo7jCow/0deFnWFbahPhW7TemZajRo/d5QBfPsDvUgugvpi86mfGkBfJZHFnZYLZIdtLxIT2EYfj7We6XK+ZZZj/lUkVkoDqiYIGGxwtUTZ93//pcKBsrFOUj1z1O4vPmE= ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968 2024-09-10T18:56:21.966 INFO:teuthology.orchestra.run.smithi047.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-10T18:56:21.967 INFO:teuthology.orchestra.run.smithi047.stdout:Adding key to root@localhost authorized_keys... 2024-09-10T18:56:21.976 INFO:teuthology.orchestra.run.smithi047.stdout:Adding host smithi047... 2024-09-10T18:56:22.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:22 smithi047 ceph-mon[29019]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:56:23.881 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:23 smithi047 ceph-mon[29019]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi047", "addr": "172.21.15.47", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:56:24.772 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:24 smithi047 ceph-mon[29019]: Deploying cephadm binary to smithi047 2024-09-10T18:56:26.474 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout Added host 'smithi047' with addr '172.21.15.47' 2024-09-10T18:56:26.474 INFO:teuthology.orchestra.run.smithi047.stdout:Deploying unmanaged mon service... 2024-09-10T18:56:27.034 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:26 smithi047 ceph-mon[29019]: from='mgr.14118 172.21.15.47:0/3692814452' entity='mgr.y' 2024-09-10T18:56:27.034 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:26 smithi047 ceph-mon[29019]: Added host smithi047 2024-09-10T18:56:27.034 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:26 smithi047 ceph-mon[29019]: from='mgr.14118 172.21.15.47:0/3692814452' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:56:27.649 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-10T18:56:27.649 INFO:teuthology.orchestra.run.smithi047.stdout:Deploying unmanaged mgr service... 2024-09-10T18:56:28.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:28 smithi047 ceph-mon[29019]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:56:28.291 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:28 smithi047 ceph-mon[29019]: Saving service mon spec with placement count:5 2024-09-10T18:56:28.291 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:28 smithi047 ceph-mon[29019]: from='mgr.14118 172.21.15.47:0/3692814452' entity='mgr.y' 2024-09-10T18:56:29.803 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-10T18:56:30.154 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:29 smithi047 ceph-mon[29019]: from='mgr.14118 172.21.15.47:0/3692814452' entity='mgr.y' 2024-09-10T18:56:30.154 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:29 smithi047 ceph-mon[29019]: from='mgr.14118 172.21.15.47:0/3692814452' entity='mgr.y' 2024-09-10T18:56:30.154 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:29 smithi047 ceph-mon[29019]: from='mgr.14118 172.21.15.47:0/3692814452' entity='mgr.y' 2024-09-10T18:56:30.941 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:30 smithi047 ceph-mon[29019]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:56:30.941 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:30 smithi047 ceph-mon[29019]: Saving service mgr spec with placement count:2 2024-09-10T18:56:30.942 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:30 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/3657669874' entity='client.admin' 2024-09-10T18:56:32.383 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:32 smithi047 ceph-mon[29019]: from='mgr.14118 172.21.15.47:0/3692814452' entity='mgr.y' 2024-09-10T18:56:32.383 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:32 smithi047 ceph-mon[29019]: from='mgr.14118 172.21.15.47:0/3692814452' entity='mgr.y' 2024-09-10T18:56:32.383 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:32 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/2182920937' entity='client.admin' 2024-09-10T18:56:32.628 INFO:teuthology.orchestra.run.smithi047.stdout:Enabling the dashboard module... 2024-09-10T18:56:33.563 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:33 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/781928853' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-10T18:56:33.564 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:33 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: ignoring --setuser ceph since I am not root 2024-09-10T18:56:33.564 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:33 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: ignoring --setgroup ceph since I am not root 2024-09-10T18:56:33.564 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:33 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T18:56:33.438+0000 7f02fb976ec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T18:56:33.564 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:33 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T18:56:33.495+0000 7f02fb976ec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T18:56:33.564 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:33 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T18:56:33.562+0000 7f02fb976ec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T18:56:33.836 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:33 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T18:56:33.690+0000 7f02fb976ec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T18:56:33.837 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:33 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T18:56:33.835+0000 7f02fb976ec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T18:56:34.379 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:34 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/781928853' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-10T18:56:34.380 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:34 smithi047 ceph-mon[29019]: mgrmap e9: y(active, since 17s) 2024-09-10T18:56:34.380 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:34 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T18:56:34.102+0000 7f02fb976ec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T18:56:34.380 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:34 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T18:56:34.165+0000 7f02fb976ec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T18:56:34.380 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:34 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T18:56:34.336+0000 7f02fb976ec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T18:56:35.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:35 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/887944944' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-10T18:56:35.540 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:35 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T18:56:35.266+0000 7f02fb976ec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T18:56:35.540 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:35 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T18:56:35.336+0000 7f02fb976ec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T18:56:35.915 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout { 2024-09-10T18:56:35.916 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-10T18:56:35.916 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-10T18:56:35.916 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-09-10T18:56:35.916 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-10T18:56:35.916 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout } 2024-09-10T18:56:35.916 INFO:teuthology.orchestra.run.smithi047.stdout:Waiting for the mgr to restart... 2024-09-10T18:56:35.916 INFO:teuthology.orchestra.run.smithi047.stdout:Waiting for mgr epoch 9... 2024-09-10T18:56:36.214 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:36 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T18:56:36.107+0000 7f02fb976ec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T18:56:36.539 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:36 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T18:56:36.226+0000 7f02fb976ec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T18:56:36.539 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:36 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T18:56:36.354+0000 7f02fb976ec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T18:56:36.539 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:36 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T18:56:36.415+0000 7f02fb976ec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T18:56:36.816 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:36 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T18:56:36.546+0000 7f02fb976ec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T18:56:36.816 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:36 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T18:56:36.615+0000 7f02fb976ec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T18:56:36.816 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:36 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T18:56:36.675+0000 7f02fb976ec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T18:56:36.817 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:36 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T18:56:36.814+0000 7f02fb976ec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T18:56:37.092 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:36 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T18:56:36.880+0000 7f02fb976ec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T18:56:37.092 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:36 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T18:56:36.947+0000 7f02fb976ec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T18:56:37.092 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:37 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T18:56:37.031+0000 7f02fb976ec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T18:56:37.092 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:37 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T18:56:37.090+0000 7f02fb976ec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T18:56:37.539 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:37 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T18:56:37.149+0000 7f02fb976ec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T18:56:38.193 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:37 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T18:56:37.926+0000 7f02fb976ec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T18:56:38.194 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:37 smithi047 ceph-mon[29019]: Active manager daemon y restarted 2024-09-10T18:56:38.194 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:37 smithi047 ceph-mon[29019]: Activating manager daemon y 2024-09-10T18:56:38.194 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:37 smithi047 ceph-mon[29019]: osdmap e3: 0 total, 0 up, 0 in 2024-09-10T18:56:38.194 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:37 smithi047 ceph-mon[29019]: mgrmap e10: y(active, starting, since 0.00325889s) 2024-09-10T18:56:38.194 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:37 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:56:38.194 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:37 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T18:56:38.194 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:37 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T18:56:38.194 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:37 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T18:56:38.195 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:37 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T18:56:38.195 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:37 smithi047 ceph-mon[29019]: Manager daemon y is now available 2024-09-10T18:56:38.195 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:37 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:56:38.195 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:37 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-10T18:56:38.195 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:37 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-10T18:56:39.842 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout { 2024-09-10T18:56:39.842 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-10T18:56:39.842 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-10T18:56:39.842 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout } 2024-09-10T18:56:39.843 INFO:teuthology.orchestra.run.smithi047.stdout:mgr epoch 9 is available 2024-09-10T18:56:39.843 INFO:teuthology.orchestra.run.smithi047.stdout:Generating a dashboard self-signed certificate... 2024-09-10T18:56:40.193 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:39 smithi047 ceph-mon[29019]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-10T18:56:40.193 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:39 smithi047 ceph-mon[29019]: mgrmap e11: y(active, since 1.00573s) 2024-09-10T18:56:40.193 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:39 smithi047 ceph-mon[29019]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-10T18:56:40.193 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:39 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:56:40.193 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:39 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T18:56:39.953+0000 7f02ac4d3640 -1 log_channel(cephadm) log [ERR] : [10/Sep/2024:18:56:39] ENGINE Error in HTTPServer.serve 2024-09-10T18:56:40.193 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:39 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: Traceback (most recent call last): 2024-09-10T18:56:40.193 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:39 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T18:56:40.194 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:39 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: self._connections.run(self.expiration_interval) 2024-09-10T18:56:40.194 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:39 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T18:56:40.194 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:39 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: self._run(expiration_interval) 2024-09-10T18:56:40.194 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:39 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T18:56:40.194 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:39 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T18:56:40.194 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:39 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T18:56:40.194 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:39 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T18:56:40.194 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:39 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T18:56:40.194 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:39 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: s = self.context.wrap_socket( 2024-09-10T18:56:40.194 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:39 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T18:56:40.194 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:39 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: return self.sslsocket_class._create( 2024-09-10T18:56:40.195 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:39 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T18:56:40.195 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:39 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: self.do_handshake() 2024-09-10T18:56:40.195 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:39 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T18:56:40.195 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:39 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: self._sslobj.do_handshake() 2024-09-10T18:56:40.195 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:39 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T18:56:40.195 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 18:56:39 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T18:56:41.126 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-10T18:56:41.126 INFO:teuthology.orchestra.run.smithi047.stdout:Creating initial admin user... 2024-09-10T18:56:41.249 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:40 smithi047 ceph-mon[29019]: [10/Sep/2024:18:56:39] ENGINE Bus STARTING 2024-09-10T18:56:41.249 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:40 smithi047 ceph-mon[29019]: [10/Sep/2024:18:56:39] ENGINE Serving on https://172.21.15.47:7150 2024-09-10T18:56:41.249 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:40 smithi047 ceph-mon[29019]: [10/Sep/2024:18:56:39] ENGINE Bus STARTED 2024-09-10T18:56:41.249 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:40 smithi047 ceph-mon[29019]: [10/Sep/2024:18:56:39] ENGINE Error in HTTPServer.serve 2024-09-10T18:56:41.249 INFO:journalctl@ceph.mon.a.smithi047.stdout: Traceback (most recent call last): 2024-09-10T18:56:41.249 INFO:journalctl@ceph.mon.a.smithi047.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T18:56:41.249 INFO:journalctl@ceph.mon.a.smithi047.stdout: self._connections.run(self.expiration_interval) 2024-09-10T18:56:41.249 INFO:journalctl@ceph.mon.a.smithi047.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T18:56:41.249 INFO:journalctl@ceph.mon.a.smithi047.stdout: self._run(expiration_interval) 2024-09-10T18:56:41.249 INFO:journalctl@ceph.mon.a.smithi047.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T18:56:41.250 INFO:journalctl@ceph.mon.a.smithi047.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T18:56:41.250 INFO:journalctl@ceph.mon.a.smithi047.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T18:56:41.250 INFO:journalctl@ceph.mon.a.smithi047.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T18:56:41.250 INFO:journalctl@ceph.mon.a.smithi047.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T18:56:41.250 INFO:journalctl@ceph.mon.a.smithi047.stdout: s = self.context.wrap_socket( 2024-09-10T18:56:41.250 INFO:journalctl@ceph.mon.a.smithi047.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T18:56:41.250 INFO:journalctl@ceph.mon.a.smithi047.stdout: return self.sslsocket_class._create( 2024-09-10T18:56:41.250 INFO:journalctl@ceph.mon.a.smithi047.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T18:56:41.250 INFO:journalctl@ceph.mon.a.smithi047.stdout: self.do_handshake() 2024-09-10T18:56:41.250 INFO:journalctl@ceph.mon.a.smithi047.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T18:56:41.250 INFO:journalctl@ceph.mon.a.smithi047.stdout: self._sslobj.do_handshake() 2024-09-10T18:56:41.250 INFO:journalctl@ceph.mon.a.smithi047.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T18:56:41.250 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:40 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:56:41.251 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:40 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:56:41.251 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:40 smithi047 ceph-mon[29019]: mgrmap e12: y(active, since 2s) 2024-09-10T18:56:41.251 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:40 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:56:41.251 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:40 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:56:42.280 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:41 smithi047 ceph-mon[29019]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:56:42.853 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$vo.KQH196LtV9.gpwt4GX.SuAy/KV64biZabG3veusw20Dm425Woi", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1725994602, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-10T18:56:42.853 INFO:teuthology.orchestra.run.smithi047.stdout:Fetching dashboard port number... 2024-09-10T18:56:43.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:43 smithi047 ceph-mon[29019]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:56:43.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:43 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:56:44.042 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 8443 2024-09-10T18:56:44.056 INFO:teuthology.orchestra.run.smithi047.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T18:56:44.056 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stdout disabled 2024-09-10T18:56:44.063 INFO:teuthology.orchestra.run.smithi047.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T18:56:44.063 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stdout inactive 2024-09-10T18:56:44.063 INFO:teuthology.orchestra.run.smithi047.stdout:firewalld.service is not enabled 2024-09-10T18:56:44.063 INFO:teuthology.orchestra.run.smithi047.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-10T18:56:44.065 INFO:teuthology.orchestra.run.smithi047.stdout:Ceph Dashboard is now available at: 2024-09-10T18:56:44.065 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:56:44.065 INFO:teuthology.orchestra.run.smithi047.stdout: URL: https://smithi047.front.sepia.ceph.com:8443/ 2024-09-10T18:56:44.065 INFO:teuthology.orchestra.run.smithi047.stdout: User: admin 2024-09-10T18:56:44.065 INFO:teuthology.orchestra.run.smithi047.stdout: Password: wqcccxa5fd 2024-09-10T18:56:44.065 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:56:44.066 INFO:teuthology.orchestra.run.smithi047.stdout:Saving cluster configuration to /var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/config directory 2024-09-10T18:56:44.066 INFO:teuthology.orchestra.run.smithi047.stdout:Enabling autotune for osd_memory_target 2024-09-10T18:56:44.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:44 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/3360310051' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-10T18:56:45.771 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:45 smithi047 ceph-mon[29019]: mgrmap e13: y(active, since 6s) 2024-09-10T18:56:46.330 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-10T18:56:46.330 INFO:teuthology.orchestra.run.smithi047.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-10T18:56:46.330 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:56:46.330 INFO:teuthology.orchestra.run.smithi047.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid d8563d7e-6fa5-11ef-bcea-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-10T18:56:46.330 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:56:46.330 INFO:teuthology.orchestra.run.smithi047.stdout:Or, if you are only running a single cluster on this host: 2024-09-10T18:56:46.330 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:56:46.330 INFO:teuthology.orchestra.run.smithi047.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-10T18:56:46.330 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:56:46.330 INFO:teuthology.orchestra.run.smithi047.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-10T18:56:46.331 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:56:46.331 INFO:teuthology.orchestra.run.smithi047.stdout: ceph telemetry on 2024-09-10T18:56:46.331 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:56:46.331 INFO:teuthology.orchestra.run.smithi047.stdout:For more information see: 2024-09-10T18:56:46.331 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:56:46.331 INFO:teuthology.orchestra.run.smithi047.stdout: https://docs.ceph.com/docs/master/mgr/telemetry/ 2024-09-10T18:56:46.331 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:56:46.331 INFO:teuthology.orchestra.run.smithi047.stdout:Bootstrap complete. 2024-09-10T18:56:46.374 INFO:tasks.cephadm:Fetching config... 2024-09-10T18:56:46.375 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-09-10T18:56:46.375 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-10T18:56:46.393 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-10T18:56:46.393 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-09-10T18:56:46.393 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-10T18:56:46.450 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-10T18:56:46.450 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-09-10T18:56:46.450 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-10T18:56:46.521 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-10T18:56:46.521 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-09-10T18:56:46.521 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-10T18:56:46.577 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-10T18:56:46.578 DEBUG:teuthology.orchestra.run.smithi047:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3paN5nDBMnsOFQW0vWC8PCBBktAtY5qw4QHN0ESfwUgnocj5QmpzSWKEVisD2cgyBM5pv7+AVJNYErwPdPYlss0QqJBJkr1vhoP62BN0VJWdxwF4ZtESE8HTLatkevft+HmA8FaIltnxtJEYA6wHGZFppaiW/p2zMwnCbxchJA8x2UW9G6hA78PBgdaj3hFV7Fv6pH/fPrWL7XoofN4rpStgGL8Z5qA3UXnzQJwxaG7ZktO3wNGjaVAtJ5aQEBJqYS4N4crMSB2qLoYOnS2X2FY7TKGAgVIT0ejBhpHeZzgVYREEp5D+2pXy94+Y55LkrQRuK26zrimTEq4+hKtu5lc9l39HWccYxajXvqrcsbhQ0JKu2XLNq9vUdbMN6TBo7jCow/0deFnWFbahPhW7TemZajRo/d5QBfPsDvUgugvpi86mfGkBfJZHFnZYLZIdtLxIT2EYfj7We6XK+ZZZj/lUkVkoDqiYIGGxwtUTZ93//pcKBsrFOUj1z1O4vPmE= ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-10T18:56:46.657 INFO:teuthology.orchestra.run.smithi047.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3paN5nDBMnsOFQW0vWC8PCBBktAtY5qw4QHN0ESfwUgnocj5QmpzSWKEVisD2cgyBM5pv7+AVJNYErwPdPYlss0QqJBJkr1vhoP62BN0VJWdxwF4ZtESE8HTLatkevft+HmA8FaIltnxtJEYA6wHGZFppaiW/p2zMwnCbxchJA8x2UW9G6hA78PBgdaj3hFV7Fv6pH/fPrWL7XoofN4rpStgGL8Z5qA3UXnzQJwxaG7ZktO3wNGjaVAtJ5aQEBJqYS4N4crMSB2qLoYOnS2X2FY7TKGAgVIT0ejBhpHeZzgVYREEp5D+2pXy94+Y55LkrQRuK26zrimTEq4+hKtu5lc9l39HWccYxajXvqrcsbhQ0JKu2XLNq9vUdbMN6TBo7jCow/0deFnWFbahPhW7TemZajRo/d5QBfPsDvUgugvpi86mfGkBfJZHFnZYLZIdtLxIT2EYfj7We6XK+ZZZj/lUkVkoDqiYIGGxwtUTZ93//pcKBsrFOUj1z1O4vPmE= ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968 2024-09-10T18:56:46.670 DEBUG:teuthology.orchestra.run.smithi149:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3paN5nDBMnsOFQW0vWC8PCBBktAtY5qw4QHN0ESfwUgnocj5QmpzSWKEVisD2cgyBM5pv7+AVJNYErwPdPYlss0QqJBJkr1vhoP62BN0VJWdxwF4ZtESE8HTLatkevft+HmA8FaIltnxtJEYA6wHGZFppaiW/p2zMwnCbxchJA8x2UW9G6hA78PBgdaj3hFV7Fv6pH/fPrWL7XoofN4rpStgGL8Z5qA3UXnzQJwxaG7ZktO3wNGjaVAtJ5aQEBJqYS4N4crMSB2qLoYOnS2X2FY7TKGAgVIT0ejBhpHeZzgVYREEp5D+2pXy94+Y55LkrQRuK26zrimTEq4+hKtu5lc9l39HWccYxajXvqrcsbhQ0JKu2XLNq9vUdbMN6TBo7jCow/0deFnWFbahPhW7TemZajRo/d5QBfPsDvUgugvpi86mfGkBfJZHFnZYLZIdtLxIT2EYfj7We6XK+ZZZj/lUkVkoDqiYIGGxwtUTZ93//pcKBsrFOUj1z1O4vPmE= ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-10T18:56:46.727 INFO:teuthology.orchestra.run.smithi149.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3paN5nDBMnsOFQW0vWC8PCBBktAtY5qw4QHN0ESfwUgnocj5QmpzSWKEVisD2cgyBM5pv7+AVJNYErwPdPYlss0QqJBJkr1vhoP62BN0VJWdxwF4ZtESE8HTLatkevft+HmA8FaIltnxtJEYA6wHGZFppaiW/p2zMwnCbxchJA8x2UW9G6hA78PBgdaj3hFV7Fv6pH/fPrWL7XoofN4rpStgGL8Z5qA3UXnzQJwxaG7ZktO3wNGjaVAtJ5aQEBJqYS4N4crMSB2qLoYOnS2X2FY7TKGAgVIT0ejBhpHeZzgVYREEp5D+2pXy94+Y55LkrQRuK26zrimTEq4+hKtu5lc9l39HWccYxajXvqrcsbhQ0JKu2XLNq9vUdbMN6TBo7jCow/0deFnWFbahPhW7TemZajRo/d5QBfPsDvUgugvpi86mfGkBfJZHFnZYLZIdtLxIT2EYfj7We6XK+ZZZj/lUkVkoDqiYIGGxwtUTZ93//pcKBsrFOUj1z1O4vPmE= ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968 2024-09-10T18:56:46.740 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8563d7e-6fa5-11ef-bcea-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-10T18:56:46.995 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:56:47.039 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:46 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/2636409863' entity='client.admin' 2024-09-10T18:56:49.801 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-10T18:56:49.801 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8563d7e-6fa5-11ef-bcea-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-10T18:56:50.028 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:50 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/1825778399' entity='client.admin' 2024-09-10T18:56:50.198 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:56:52.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:52 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:56:52.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:52 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:56:52.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:52 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi047", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:56:52.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:52 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:56:52.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:52 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:56:52.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:52 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:56:53.306 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi149 2024-09-10T18:56:53.306 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-09-10T18:56:53.306 DEBUG:teuthology.orchestra.run.smithi149:> dd of=/etc/ceph/ceph.conf 2024-09-10T18:56:53.325 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-09-10T18:56:53.325 DEBUG:teuthology.orchestra.run.smithi149:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:56:53.383 INFO:tasks.cephadm:Adding host smithi149 to orchestrator... 2024-09-10T18:56:53.383 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8563d7e-6fa5-11ef-bcea-c7b262605968 -- ceph orch host add smithi149 2024-09-10T18:56:53.639 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:56:53.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:53 smithi047 ceph-mon[29019]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:56:53.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:53 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:56:53.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:53 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:56:53.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:53 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:56:53.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:53 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:56:53.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:53 smithi047 ceph-mon[29019]: Updating smithi047:/etc/ceph/ceph.conf 2024-09-10T18:56:53.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:53 smithi047 ceph-mon[29019]: Updating smithi047:/var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:56:53.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:53 smithi047 ceph-mon[29019]: Updating smithi047:/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:56:53.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:53 smithi047 ceph-mon[29019]: Updating smithi047:/var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T18:56:53.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:53 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:56:53.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:53 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:56:53.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:53 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:56:55.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:55 smithi047 ceph-mon[29019]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi149", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:56:56.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:56 smithi047 ceph-mon[29019]: Deploying cephadm binary to smithi149 2024-09-10T18:56:58.527 INFO:teuthology.orchestra.run.smithi047.stdout:Added host 'smithi149' with addr '172.21.15.149' 2024-09-10T18:56:59.064 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8563d7e-6fa5-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-10T18:56:59.088 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:58 smithi047 ceph-mon[29019]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:56:59.088 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:58 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:56:59.088 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:58 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:56:59.088 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:56:58 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:56:59.312 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:57:00.354 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:00 smithi047 ceph-mon[29019]: Added host smithi149 2024-09-10T18:57:00.354 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:00 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:00.354 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:00 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:00.989 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T18:57:00.989 INFO:teuthology.orchestra.run.smithi047.stdout:[{"addr": "172.21.15.47", "hostname": "smithi047", "labels": [], "status": ""}, {"addr": "172.21.15.149", "hostname": "smithi149", "labels": [], "status": ""}] 2024-09-10T18:57:01.289 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:01 smithi047 ceph-mon[29019]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:57:01.620 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-10T18:57:01.621 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8563d7e-6fa5-11ef-bcea-c7b262605968 -- ceph osd crush tunables default 2024-09-10T18:57:01.910 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:57:02.289 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:02 smithi047 ceph-mon[29019]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:57:03.135 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:03 smithi047 ceph-mon[29019]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:57:03.135 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:03 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:03.135 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:03 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:03.135 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:03 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:03.135 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:03 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:03.135 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:03 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:57:03.135 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:03 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:57:03.135 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:03 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:57:04.110 INFO:teuthology.orchestra.run.smithi047.stderr:adjusted tunables profile to default 2024-09-10T18:57:04.408 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:04 smithi047 ceph-mon[29019]: Updating smithi149:/etc/ceph/ceph.conf 2024-09-10T18:57:04.408 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:04 smithi047 ceph-mon[29019]: Updating smithi149:/var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:57:04.408 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:04 smithi047 ceph-mon[29019]: Updating smithi149:/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:57:04.408 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:04 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/3105283635' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-10T18:57:04.408 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:04 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:04.408 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:04 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:04.408 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:04 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:04.714 INFO:tasks.cephadm:Adding mon.a on smithi047 2024-09-10T18:57:04.714 INFO:tasks.cephadm:Adding mon.c on smithi047 2024-09-10T18:57:04.714 INFO:tasks.cephadm:Adding mon.b on smithi149 2024-09-10T18:57:04.714 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8563d7e-6fa5-11ef-bcea-c7b262605968 -- ceph orch apply mon '3;smithi047:172.21.15.47=a;smithi047:[v2:172.21.15.47:3301,v1:172.21.15.47:6790]=c;smithi149:172.21.15.149=b' 2024-09-10T18:57:04.958 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:57:05.427 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:05 smithi047 ceph-mon[29019]: Updating smithi149:/var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T18:57:05.427 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:05 smithi047 ceph-mon[29019]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:57:05.427 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:05 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/3105283635' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-10T18:57:05.427 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:05 smithi047 ceph-mon[29019]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T18:57:06.659 INFO:teuthology.orchestra.run.smithi149.stdout:Scheduled mon update... 2024-09-10T18:57:07.230 DEBUG:teuthology.orchestra.run.smithi047:mon.c> sudo journalctl -f -n 0 -u ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968@mon.c.service 2024-09-10T18:57:07.237 DEBUG:teuthology.orchestra.run.smithi149:mon.b> sudo journalctl -f -n 0 -u ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968@mon.b.service 2024-09-10T18:57:07.239 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-10T18:57:07.240 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8563d7e-6fa5-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T18:57:07.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:07 smithi047 ceph-mon[29019]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:57:07.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:07 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:07.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:07 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:57:07.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:07 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:57:07.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:07 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:57:07.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:07 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:07.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:07 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:57:07.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:07 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:57:08.073 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:57:08.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:08 smithi047 ceph-mon[29019]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi047:172.21.15.47=a;smithi047:[v2:172.21.15.47:3301,v1:172.21.15.47:6790]=c;smithi149:172.21.15.149=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:57:08.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:08 smithi047 ceph-mon[29019]: Saving service mon spec with placement smithi047:172.21.15.47=a;smithi047:[v2:172.21.15.47:3301,v1:172.21.15.47:6790]=c;smithi149:172.21.15.149=b;count:3 2024-09-10T18:57:08.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:08 smithi047 ceph-mon[29019]: Deploying daemon mon.b on smithi149 2024-09-10T18:57:09.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:09 smithi047 ceph-mon[29019]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:57:10.269 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T18:57:10.269 INFO:teuthology.orchestra.run.smithi149.stdout:{"epoch":1,"fsid":"d8563d7e-6fa5-11ef-bcea-c7b262605968","modified":"2024-09-10T18:55:41.695745Z","created":"2024-09-10T18:55:41.695745Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-10T18:57:10.270 INFO:teuthology.orchestra.run.smithi149.stderr:dumped monmap epoch 1 2024-09-10T18:57:11.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:11 smithi047 ceph-mon[29019]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:57:11.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:11 smithi047 ceph-mon[29019]: from='client.? 172.21.15.149:0/2636908548' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T18:57:12.309 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 podman[33211]: 2024-09-10 18:57:12.308175969 +0000 UTC m=+0.326088209 container init fb64053d26660f7d8ccf6937602dafd33b7b9dd44b349546da8d12af455c6280 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mon-b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, ceph=True) 2024-09-10T18:57:12.382 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-10T18:57:12.383 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8563d7e-6fa5-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T18:57:12.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 podman[33211]: 2024-09-10 18:57:12.311281604 +0000 UTC m=+0.329193847 container start fb64053d26660f7d8ccf6937602dafd33b7b9dd44b349546da8d12af455c6280 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mon-b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T18:57:12.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: set uid:gid to 167:167 (ceph:ceph) 2024-09-10T18:57:12.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: ceph version 17.2.7-1606-g5be57395 (5be57395ea235535910512ae73ba8e7ff25fe588) quincy (stable), process ceph-mon, pid 2 2024-09-10T18:57:12.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: pidfile_write: ignore empty --pid-file 2024-09-10T18:57:12.594 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: load: jerasure load: lrc 2024-09-10T18:57:12.594 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: RocksDB version: 6.15.5 2024-09-10T18:57:12.594 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-09-10T18:57:12.594 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Compile date Sep 9 2024 2024-09-10T18:57:12.594 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: DB SUMMARY 2024-09-10T18:57:12.594 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: DB Session ID: 3WX48K497KE9ZUML6YRN 2024-09-10T18:57:12.594 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: CURRENT file: CURRENT 2024-09-10T18:57:12.594 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: IDENTITY file: IDENTITY 2024-09-10T18:57:12.594 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: MANIFEST file: MANIFEST-000003 size: 57 Bytes 2024-09-10T18:57:12.594 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-09-10T18:57:12.594 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 511 ; 2024-09-10T18:57:12.594 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.error_if_exists: 0 2024-09-10T18:57:12.595 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.create_if_missing: 0 2024-09-10T18:57:12.595 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.paranoid_checks: 1 2024-09-10T18:57:12.595 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-10T18:57:12.595 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.env: 0x55d8815ea440 2024-09-10T18:57:12.595 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.fs: Posix File System 2024-09-10T18:57:12.595 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.info_log: 0x55d8826d0ec0 2024-09-10T18:57:12.595 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.max_file_opening_threads: 16 2024-09-10T18:57:12.595 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.statistics: (nil) 2024-09-10T18:57:12.595 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.use_fsync: 0 2024-09-10T18:57:12.595 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.max_log_file_size: 0 2024-09-10T18:57:12.595 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-10T18:57:12.595 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-10T18:57:12.595 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.keep_log_file_num: 1000 2024-09-10T18:57:12.596 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.recycle_log_file_num: 0 2024-09-10T18:57:12.596 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.allow_fallocate: 1 2024-09-10T18:57:12.596 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.allow_mmap_reads: 0 2024-09-10T18:57:12.596 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.allow_mmap_writes: 0 2024-09-10T18:57:12.596 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.use_direct_reads: 0 2024-09-10T18:57:12.596 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-10T18:57:12.596 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.create_missing_column_families: 0 2024-09-10T18:57:12.596 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.db_log_dir: 2024-09-10T18:57:12.596 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-b/store.db 2024-09-10T18:57:12.596 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-10T18:57:12.597 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-10T18:57:12.597 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-10T18:57:12.597 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-10T18:57:12.597 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-10T18:57:12.597 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-10T18:57:12.597 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.advise_random_on_open: 1 2024-09-10T18:57:12.597 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.db_write_buffer_size: 0 2024-09-10T18:57:12.598 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.write_buffer_manager: 0x55d88278e7e0 2024-09-10T18:57:12.598 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-10T18:57:12.598 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-09-10T18:57:12.598 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-10T18:57:12.598 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-10T18:57:12.598 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.rate_limiter: (nil) 2024-09-10T18:57:12.599 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-10T18:57:12.599 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.wal_recovery_mode: 2 2024-09-10T18:57:12.599 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.enable_thread_tracking: 0 2024-09-10T18:57:12.599 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.enable_pipelined_write: 0 2024-09-10T18:57:12.599 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.unordered_write: 0 2024-09-10T18:57:12.599 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-10T18:57:12.599 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-10T18:57:12.599 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-10T18:57:12.599 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-10T18:57:12.600 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.row_cache: None 2024-09-10T18:57:12.600 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.wal_filter: None 2024-09-10T18:57:12.600 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-10T18:57:12.600 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.allow_ingest_behind: 0 2024-09-10T18:57:12.600 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.preserve_deletes: 0 2024-09-10T18:57:12.600 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.two_write_queues: 0 2024-09-10T18:57:12.600 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.manual_wal_flush: 0 2024-09-10T18:57:12.600 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.atomic_flush: 0 2024-09-10T18:57:12.600 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-10T18:57:12.600 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-10T18:57:12.600 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-10T18:57:12.602 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.log_readahead_size: 0 2024-09-10T18:57:12.602 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-10T18:57:12.602 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.best_efforts_recovery: 0 2024-09-10T18:57:12.603 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-10T18:57:12.603 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-10T18:57:12.603 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.allow_data_in_errors: 0 2024-09-10T18:57:12.603 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.db_host_id: __hostname__ 2024-09-10T18:57:12.603 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.max_background_jobs: 2 2024-09-10T18:57:12.603 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.max_background_compactions: -1 2024-09-10T18:57:12.603 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.max_subcompactions: 1 2024-09-10T18:57:12.603 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-10T18:57:12.604 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-10T18:57:12.604 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-10T18:57:12.604 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.max_total_wal_size: 0 2024-09-10T18:57:12.604 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-10T18:57:12.604 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-10T18:57:12.604 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-10T18:57:12.604 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-10T18:57:12.604 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.max_open_files: -1 2024-09-10T18:57:12.604 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.bytes_per_sync: 0 2024-09-10T18:57:12.604 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-10T18:57:12.604 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-10T18:57:12.604 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.compaction_readahead_size: 0 2024-09-10T18:57:12.605 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.max_background_flushes: -1 2024-09-10T18:57:12.605 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Compression algorithms supported: 2024-09-10T18:57:12.605 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: kZSTD supported: 0 2024-09-10T18:57:12.605 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: kXpressCompression supported: 0 2024-09-10T18:57:12.605 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: kBZip2Compression supported: 0 2024-09-10T18:57:12.605 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-10T18:57:12.605 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: kLZ4Compression supported: 1 2024-09-10T18:57:12.605 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: kZlibCompression supported: 1 2024-09-10T18:57:12.605 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: kLZ4HCCompression supported: 1 2024-09-10T18:57:12.605 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: kSnappyCompression supported: 1 2024-09-10T18:57:12.605 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-10T18:57:12.605 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000003 2024-09-10T18:57:12.606 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2024-09-10T18:57:12.606 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-10T18:57:12.606 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.merge_operator: 2024-09-10T18:57:12.606 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.compaction_filter: None 2024-09-10T18:57:12.606 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.compaction_filter_factory: None 2024-09-10T18:57:12.606 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.sst_partitioner_factory: None 2024-09-10T18:57:12.606 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-10T18:57:12.606 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-10T18:57:12.606 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55d882677a38) 2024-09-10T18:57:12.606 INFO:journalctl@ceph.mon.b.smithi149.stdout: cache_index_and_filter_blocks: 1 2024-09-10T18:57:12.606 INFO:journalctl@ceph.mon.b.smithi149.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-10T18:57:12.606 INFO:journalctl@ceph.mon.b.smithi149.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-10T18:57:12.606 INFO:journalctl@ceph.mon.b.smithi149.stdout: pin_top_level_index_and_filter: 1 2024-09-10T18:57:12.606 INFO:journalctl@ceph.mon.b.smithi149.stdout: index_type: 0 2024-09-10T18:57:12.607 INFO:journalctl@ceph.mon.b.smithi149.stdout: data_block_index_type: 0 2024-09-10T18:57:12.607 INFO:journalctl@ceph.mon.b.smithi149.stdout: index_shortening: 1 2024-09-10T18:57:12.607 INFO:journalctl@ceph.mon.b.smithi149.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-10T18:57:12.607 INFO:journalctl@ceph.mon.b.smithi149.stdout: hash_index_allow_collision: 1 2024-09-10T18:57:12.607 INFO:journalctl@ceph.mon.b.smithi149.stdout: checksum: 1 2024-09-10T18:57:12.607 INFO:journalctl@ceph.mon.b.smithi149.stdout: no_block_cache: 0 2024-09-10T18:57:12.607 INFO:journalctl@ceph.mon.b.smithi149.stdout: block_cache: 0x55d8826ad1f0 2024-09-10T18:57:12.607 INFO:journalctl@ceph.mon.b.smithi149.stdout: block_cache_name: BinnedLRUCache 2024-09-10T18:57:12.607 INFO:journalctl@ceph.mon.b.smithi149.stdout: block_cache_options: 2024-09-10T18:57:12.607 INFO:journalctl@ceph.mon.b.smithi149.stdout: capacity : 536870912 2024-09-10T18:57:12.608 INFO:journalctl@ceph.mon.b.smithi149.stdout: num_shard_bits : 4 2024-09-10T18:57:12.608 INFO:journalctl@ceph.mon.b.smithi149.stdout: strict_capacity_limit : 0 2024-09-10T18:57:12.608 INFO:journalctl@ceph.mon.b.smithi149.stdout: high_pri_pool_ratio: 0.000 2024-09-10T18:57:12.608 INFO:journalctl@ceph.mon.b.smithi149.stdout: block_cache_compressed: (nil) 2024-09-10T18:57:12.608 INFO:journalctl@ceph.mon.b.smithi149.stdout: persistent_cache: (nil) 2024-09-10T18:57:12.608 INFO:journalctl@ceph.mon.b.smithi149.stdout: block_size: 4096 2024-09-10T18:57:12.609 INFO:journalctl@ceph.mon.b.smithi149.stdout: block_size_deviation: 10 2024-09-10T18:57:12.609 INFO:journalctl@ceph.mon.b.smithi149.stdout: block_restart_interval: 16 2024-09-10T18:57:12.609 INFO:journalctl@ceph.mon.b.smithi149.stdout: index_block_restart_interval: 1 2024-09-10T18:57:12.609 INFO:journalctl@ceph.mon.b.smithi149.stdout: metadata_block_size: 4096 2024-09-10T18:57:12.609 INFO:journalctl@ceph.mon.b.smithi149.stdout: partition_filters: 0 2024-09-10T18:57:12.609 INFO:journalctl@ceph.mon.b.smithi149.stdout: use_delta_encoding: 1 2024-09-10T18:57:12.609 INFO:journalctl@ceph.mon.b.smithi149.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2024-09-10T18:57:12.609 INFO:journalctl@ceph.mon.b.smithi149.stdout: whole_key_filtering: 1 2024-09-10T18:57:12.609 INFO:journalctl@ceph.mon.b.smithi149.stdout: verify_compression: 0 2024-09-10T18:57:12.609 INFO:journalctl@ceph.mon.b.smithi149.stdout: read_amp_bytes_per_bit: 0 2024-09-10T18:57:12.610 INFO:journalctl@ceph.mon.b.smithi149.stdout: format_version: 4 2024-09-10T18:57:12.610 INFO:journalctl@ceph.mon.b.smithi149.stdout: enable_index_compression: 1 2024-09-10T18:57:12.610 INFO:journalctl@ceph.mon.b.smithi149.stdout: block_align: 0 2024-09-10T18:57:12.610 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.write_buffer_size: 33554432 2024-09-10T18:57:12.610 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.max_write_buffer_number: 2 2024-09-10T18:57:12.610 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.compression: NoCompression 2024-09-10T18:57:12.610 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.bottommost_compression: Disabled 2024-09-10T18:57:12.610 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.prefix_extractor: nullptr 2024-09-10T18:57:12.610 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-10T18:57:12.610 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.num_levels: 7 2024-09-10T18:57:12.610 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-10T18:57:12.610 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-10T18:57:12.610 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-10T18:57:12.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-10T18:57:12.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-10T18:57:12.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-10T18:57:12.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-10T18:57:12.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-10T18:57:12.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-10T18:57:12.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-10T18:57:12.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-10T18:57:12.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.compression_opts.level: 32767 2024-09-10T18:57:12.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.compression_opts.strategy: 0 2024-09-10T18:57:12.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-10T18:57:12.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-10T18:57:12.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-10T18:57:12.612 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.compression_opts.enabled: false 2024-09-10T18:57:12.612 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-10T18:57:12.612 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-10T18:57:12.612 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-10T18:57:12.612 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.target_file_size_base: 67108864 2024-09-10T18:57:12.612 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-10T18:57:12.612 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-10T18:57:12.612 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-10T18:57:12.612 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-10T18:57:12.612 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-10T18:57:12.612 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-10T18:57:12.612 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-10T18:57:12.613 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-10T18:57:12.613 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-10T18:57:12.613 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-10T18:57:12.613 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-10T18:57:12.613 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-10T18:57:12.613 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-10T18:57:12.614 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.arena_block_size: 4194304 2024-09-10T18:57:12.614 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-10T18:57:12.614 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-10T18:57:12.614 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-09-10T18:57:12.614 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.disable_auto_compactions: 0 2024-09-10T18:57:12.614 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-10T18:57:12.614 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-10T18:57:12.614 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-10T18:57:12.614 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-10T18:57:12.614 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-10T18:57:12.614 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-10T18:57:12.614 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-10T18:57:12.614 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-10T18:57:12.615 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-10T18:57:12.615 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-10T18:57:12.615 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-10T18:57:12.615 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.inplace_update_support: 0 2024-09-10T18:57:12.615 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-10T18:57:12.615 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-10T18:57:12.615 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-10T18:57:12.615 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-10T18:57:12.615 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.bloom_locality: 0 2024-09-10T18:57:12.616 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.max_successive_merges: 0 2024-09-10T18:57:12.616 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-10T18:57:12.616 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.paranoid_file_checks: 0 2024-09-10T18:57:12.616 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.force_consistency_checks: 1 2024-09-10T18:57:12.616 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.report_bg_io_stats: 0 2024-09-10T18:57:12.616 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.ttl: 2592000 2024-09-10T18:57:12.616 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-10T18:57:12.616 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.enable_blob_files: false 2024-09-10T18:57:12.616 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.min_blob_size: 0 2024-09-10T18:57:12.616 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.blob_file_size: 268435456 2024-09-10T18:57:12.616 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-10T18:57:12.616 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-10T18:57:12.616 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-10T18:57:12.616 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000003 succeeded,manifest_file_number is 3, next_file_number is 5, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-10T18:57:12.617 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 0 2024-09-10T18:57:12.617 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: [db/version_set.cc:4082] Creating manifest 7 2024-09-10T18:57:12.617 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725994632335674, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-10T18:57:12.617 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #4 mode 2 2024-09-10T18:57:12.617 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: [table/block_based/filter_policy.cc:991] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-09-10T18:57:12.617 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725994632335988, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1571, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1725994632, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "9ec792ad-8c0b-4eff-bde7-f06ca75ce90b", "db_session_id": "3WX48K497KE9ZUML6YRN"}} 2024-09-10T18:57:12.617 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: [db/version_set.cc:4082] Creating manifest 9 2024-09-10T18:57:12.617 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725994632336304, "job": 1, "event": "recovery_finished"} 2024-09-10T18:57:12.617 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-10T18:57:12.618 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x55d8826e0700 2024-09-10T18:57:12.618 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: DB pointer 0x55d8826b0000 2024-09-10T18:57:12.618 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2024-09-10T18:57:12.618 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: rocksdb: [db/db_impl/db_impl.cc:903] 2024-09-10T18:57:12.618 INFO:journalctl@ceph.mon.b.smithi149.stdout: ** DB Stats ** 2024-09-10T18:57:12.618 INFO:journalctl@ceph.mon.b.smithi149.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T18:57:12.619 INFO:journalctl@ceph.mon.b.smithi149.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-10T18:57:12.619 INFO:journalctl@ceph.mon.b.smithi149.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T18:57:12.619 INFO:journalctl@ceph.mon.b.smithi149.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T18:57:12.619 INFO:journalctl@ceph.mon.b.smithi149.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-10T18:57:12.619 INFO:journalctl@ceph.mon.b.smithi149.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-09-10T18:57:12.619 INFO:journalctl@ceph.mon.b.smithi149.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T18:57:12.619 INFO:journalctl@ceph.mon.b.smithi149.stdout: 2024-09-10T18:57:12.619 INFO:journalctl@ceph.mon.b.smithi149.stdout: ** Compaction Stats [default] ** 2024-09-10T18:57:12.619 INFO:journalctl@ceph.mon.b.smithi149.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T18:57:12.619 INFO:journalctl@ceph.mon.b.smithi149.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T18:57:12.619 INFO:journalctl@ceph.mon.b.smithi149.stdout: L0 1/0 1.53 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.0 0.00 0.00 1 0.000 0 0 2024-09-10T18:57:12.619 INFO:journalctl@ceph.mon.b.smithi149.stdout: Sum 1/0 1.53 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.0 0.00 0.00 1 0.000 0 0 2024-09-10T18:57:12.619 INFO:journalctl@ceph.mon.b.smithi149.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.0 0.00 0.00 1 0.000 0 0 2024-09-10T18:57:12.619 INFO:journalctl@ceph.mon.b.smithi149.stdout: 2024-09-10T18:57:12.620 INFO:journalctl@ceph.mon.b.smithi149.stdout: ** Compaction Stats [default] ** 2024-09-10T18:57:12.620 INFO:journalctl@ceph.mon.b.smithi149.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T18:57:12.620 INFO:journalctl@ceph.mon.b.smithi149.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T18:57:12.620 INFO:journalctl@ceph.mon.b.smithi149.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 5.0 0.00 0.00 1 0.000 0 0 2024-09-10T18:57:12.620 INFO:journalctl@ceph.mon.b.smithi149.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T18:57:12.620 INFO:journalctl@ceph.mon.b.smithi149.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-10T18:57:12.620 INFO:journalctl@ceph.mon.b.smithi149.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-10T18:57:12.620 INFO:journalctl@ceph.mon.b.smithi149.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-10T18:57:12.620 INFO:journalctl@ceph.mon.b.smithi149.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-10T18:57:12.620 INFO:journalctl@ceph.mon.b.smithi149.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-10T18:57:12.620 INFO:journalctl@ceph.mon.b.smithi149.stdout: Cumulative compaction: 0.00 GB write, 0.55 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T18:57:12.620 INFO:journalctl@ceph.mon.b.smithi149.stdout: Interval compaction: 0.00 GB write, 0.55 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T18:57:12.620 INFO:journalctl@ceph.mon.b.smithi149.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-10T18:57:12.621 INFO:journalctl@ceph.mon.b.smithi149.stdout: 2024-09-10T18:57:12.621 INFO:journalctl@ceph.mon.b.smithi149.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-10T18:57:12.621 INFO:journalctl@ceph.mon.b.smithi149.stdout: 2024-09-10T18:57:12.621 INFO:journalctl@ceph.mon.b.smithi149.stdout: ** Compaction Stats [default] ** 2024-09-10T18:57:12.621 INFO:journalctl@ceph.mon.b.smithi149.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T18:57:12.621 INFO:journalctl@ceph.mon.b.smithi149.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T18:57:12.621 INFO:journalctl@ceph.mon.b.smithi149.stdout: L0 1/0 1.53 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.0 0.00 0.00 1 0.000 0 0 2024-09-10T18:57:12.621 INFO:journalctl@ceph.mon.b.smithi149.stdout: Sum 1/0 1.53 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.0 0.00 0.00 1 0.000 0 0 2024-09-10T18:57:12.621 INFO:journalctl@ceph.mon.b.smithi149.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-09-10T18:57:12.621 INFO:journalctl@ceph.mon.b.smithi149.stdout: 2024-09-10T18:57:12.621 INFO:journalctl@ceph.mon.b.smithi149.stdout: ** Compaction Stats [default] ** 2024-09-10T18:57:12.621 INFO:journalctl@ceph.mon.b.smithi149.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T18:57:12.622 INFO:journalctl@ceph.mon.b.smithi149.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T18:57:12.622 INFO:journalctl@ceph.mon.b.smithi149.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 5.0 0.00 0.00 1 0.000 0 0 2024-09-10T18:57:12.622 INFO:journalctl@ceph.mon.b.smithi149.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T18:57:12.622 INFO:journalctl@ceph.mon.b.smithi149.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-10T18:57:12.622 INFO:journalctl@ceph.mon.b.smithi149.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-10T18:57:12.622 INFO:journalctl@ceph.mon.b.smithi149.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-10T18:57:12.622 INFO:journalctl@ceph.mon.b.smithi149.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-10T18:57:12.623 INFO:journalctl@ceph.mon.b.smithi149.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-10T18:57:12.623 INFO:journalctl@ceph.mon.b.smithi149.stdout: Cumulative compaction: 0.00 GB write, 0.55 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T18:57:12.623 INFO:journalctl@ceph.mon.b.smithi149.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T18:57:12.623 INFO:journalctl@ceph.mon.b.smithi149.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-10T18:57:12.623 INFO:journalctl@ceph.mon.b.smithi149.stdout: 2024-09-10T18:57:12.623 INFO:journalctl@ceph.mon.b.smithi149.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-10T18:57:12.624 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-09-10T18:57:12.624 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: using public_addr v2:172.21.15.149:0/0 -> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] 2024-09-10T18:57:12.624 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: starting mon.b rank -1 at public addrs [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] at bind addrs [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid d8563d7e-6fa5-11ef-bcea-c7b262605968 2024-09-10T18:57:12.624 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: mon.b@-1(???) e0 preinit fsid d8563d7e-6fa5-11ef-bcea-c7b262605968 2024-09-10T18:57:12.624 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: mon.b@-1(synchronizing).mds e1 new map 2024-09-10T18:57:12.625 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: mon.b@-1(synchronizing).mds e1 print_map 2024-09-10T18:57:12.625 INFO:journalctl@ceph.mon.b.smithi149.stdout: e1 2024-09-10T18:57:12.625 INFO:journalctl@ceph.mon.b.smithi149.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T18:57:12.625 INFO:journalctl@ceph.mon.b.smithi149.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T18:57:12.625 INFO:journalctl@ceph.mon.b.smithi149.stdout: legacy client fscid: -1 2024-09-10T18:57:12.625 INFO:journalctl@ceph.mon.b.smithi149.stdout: 2024-09-10T18:57:12.625 INFO:journalctl@ceph.mon.b.smithi149.stdout: No filesystems configured 2024-09-10T18:57:12.625 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-10T18:57:12.625 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-10T18:57:12.626 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-10T18:57:12.626 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-10T18:57:12.626 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-10T18:57:12.626 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-10T18:57:12.626 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-10T18:57:12.627 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T18:57:12.627 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T18:57:12.627 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T18:57:12.627 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:57:12.627 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:57:12.627 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:12.628 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:12.628 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:12.628 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:12.628 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:57:12.628 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:57:12.628 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:57:12.629 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: Updating smithi149:/etc/ceph/ceph.conf 2024-09-10T18:57:12.629 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: Updating smithi149:/var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:57:12.629 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: Updating smithi149:/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:57:12.629 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/3105283635' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-10T18:57:12.629 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:12.629 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:12.629 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:12.629 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: Updating smithi149:/var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T18:57:12.629 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:57:12.629 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/3105283635' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-10T18:57:12.630 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T18:57:12.630 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:57:12.630 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:12.630 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:57:12.630 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:57:12.630 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:57:12.630 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:12.630 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:57:12.630 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:57:12.630 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi047:172.21.15.47=a;smithi047:[v2:172.21.15.47:3301,v1:172.21.15.47:6790]=c;smithi149:172.21.15.149=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:57:12.630 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: Saving service mon spec with placement smithi047:172.21.15.47=a;smithi047:[v2:172.21.15.47:3301,v1:172.21.15.47:6790]=c;smithi149:172.21.15.149=b;count:3 2024-09-10T18:57:12.630 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: Deploying daemon mon.b on smithi149 2024-09-10T18:57:12.631 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:57:12.631 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:57:12.631 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: from='client.? 172.21.15.149:0/2636908548' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T18:57:12.631 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 ceph-mon[33230]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-10T18:57:12.631 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 bash[33211]: fb64053d26660f7d8ccf6937602dafd33b7b9dd44b349546da8d12af455c6280 2024-09-10T18:57:12.631 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:12 smithi149 systemd[1]: Started Ceph mon.b for d8563d7e-6fa5-11ef-bcea-c7b262605968. 2024-09-10T18:57:12.645 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:57:17.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:17 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:57:17.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:17 smithi047 ceph-mon[29019]: mon.a calling monitor election 2024-09-10T18:57:17.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:17 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:57:17.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:17 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:57:17.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:17 smithi047 ceph-mon[29019]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:57:17.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:17 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:57:17.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:17 smithi047 ceph-mon[29019]: mon.b calling monitor election 2024-09-10T18:57:17.791 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:17 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:57:17.791 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:17 smithi047 ceph-mon[29019]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:57:17.791 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:17 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:57:17.791 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:17 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:57:17.791 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:17 smithi047 ceph-mon[29019]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-10T18:57:17.791 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:17 smithi047 ceph-mon[29019]: monmap e2: 2 mons at {a=[v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0],b=[v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-10T18:57:17.791 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:17 smithi047 ceph-mon[29019]: fsmap 2024-09-10T18:57:17.791 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:17 smithi047 ceph-mon[29019]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T18:57:17.791 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:17 smithi047 ceph-mon[29019]: mgrmap e13: y(active, since 39s) 2024-09-10T18:57:17.791 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:17 smithi047 ceph-mon[29019]: overall HEALTH_OK 2024-09-10T18:57:17.791 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:17 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:17.791 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:17 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:17.791 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:17 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:17.792 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:17 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:57:17.792 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:17 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:57:17.843 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:17 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:57:17.843 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:17 smithi149 ceph-mon[33230]: mon.a calling monitor election 2024-09-10T18:57:17.843 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:17 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:57:17.843 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:17 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:57:17.843 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:17 smithi149 ceph-mon[33230]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:57:17.843 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:17 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:57:17.843 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:17 smithi149 ceph-mon[33230]: mon.b calling monitor election 2024-09-10T18:57:17.844 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:17 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:57:17.844 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:17 smithi149 ceph-mon[33230]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:57:17.844 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:17 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:57:17.844 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:17 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:57:17.844 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:17 smithi149 ceph-mon[33230]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-10T18:57:17.844 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:17 smithi149 ceph-mon[33230]: monmap e2: 2 mons at {a=[v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0],b=[v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-10T18:57:17.844 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:17 smithi149 ceph-mon[33230]: fsmap 2024-09-10T18:57:17.844 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:17 smithi149 ceph-mon[33230]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T18:57:17.844 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:17 smithi149 ceph-mon[33230]: mgrmap e13: y(active, since 39s) 2024-09-10T18:57:17.844 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:17 smithi149 ceph-mon[33230]: overall HEALTH_OK 2024-09-10T18:57:17.844 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:17 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:17.844 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:17 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:17.844 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:17 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:17.844 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:17 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:57:17.844 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:17 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:57:17.916 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T18:57:17.916 INFO:teuthology.orchestra.run.smithi149.stdout:{"epoch":2,"fsid":"d8563d7e-6fa5-11ef-bcea-c7b262605968","modified":"2024-09-10T18:57:12.361003Z","created":"2024-09-10T18:55:41.695745Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:3300","nonce":0},{"type":"v1","addr":"172.21.15.149:6789","nonce":0}]},"addr":"172.21.15.149:6789/0","public_addr":"172.21.15.149:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-10T18:57:17.916 INFO:teuthology.orchestra.run.smithi149.stderr:dumped monmap epoch 2 2024-09-10T18:57:18.487 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:18 smithi149 ceph-mon[33230]: Deploying daemon mon.c on smithi047 2024-09-10T18:57:18.487 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:18 smithi149 ceph-mon[33230]: from='client.? 172.21.15.149:0/221376471' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T18:57:18.487 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:18 smithi149 ceph-mon[33230]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:57:18.487 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:18 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:57:18.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:18 smithi047 ceph-mon[29019]: Deploying daemon mon.c on smithi047 2024-09-10T18:57:18.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:18 smithi047 ceph-mon[29019]: from='client.? 172.21.15.149:0/221376471' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T18:57:18.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:18 smithi047 ceph-mon[29019]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:57:18.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:18 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:57:19.512 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-10T18:57:19.512 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8563d7e-6fa5-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T18:57:19.759 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:57:21.039 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:20 smithi047 systemd[1]: Started Ceph mon.c for d8563d7e-6fa5-11ef-bcea-c7b262605968. 2024-09-10T18:57:26.040 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:25 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:57:26.040 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:25 smithi047 ceph-mon[39856]: mon.a calling monitor election 2024-09-10T18:57:26.040 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:25 smithi047 ceph-mon[39856]: mon.b calling monitor election 2024-09-10T18:57:26.040 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:25 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:57:26.040 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:25 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:57:26.040 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:25 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:57:26.041 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:25 smithi047 ceph-mon[39856]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:57:26.041 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:25 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:57:26.041 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:25 smithi047 ceph-mon[39856]: mon.c calling monitor election 2024-09-10T18:57:26.041 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:25 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:57:26.041 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:25 smithi047 ceph-mon[39856]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:57:26.041 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:25 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:57:26.041 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:25 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:57:26.041 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:25 smithi047 ceph-mon[39856]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-10T18:57:26.041 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:25 smithi047 ceph-mon[39856]: monmap e3: 3 mons at {a=[v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0],b=[v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0],c=[v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-10T18:57:26.041 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:25 smithi047 ceph-mon[39856]: fsmap 2024-09-10T18:57:26.041 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:25 smithi047 ceph-mon[39856]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T18:57:26.041 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:25 smithi047 ceph-mon[39856]: mgrmap e13: y(active, since 47s) 2024-09-10T18:57:26.042 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:25 smithi047 ceph-mon[39856]: overall HEALTH_OK 2024-09-10T18:57:26.042 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:25 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:26.042 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:25 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:26.042 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:25 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:26.042 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:25 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:26.042 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:25 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:57:26.042 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:25 smithi047 ceph-mon[29019]: mon.a calling monitor election 2024-09-10T18:57:26.042 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:25 smithi047 ceph-mon[29019]: mon.b calling monitor election 2024-09-10T18:57:26.043 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:25 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:57:26.043 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:25 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:57:26.043 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:25 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:57:26.043 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:25 smithi047 ceph-mon[29019]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:57:26.043 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:25 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:57:26.043 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:25 smithi047 ceph-mon[29019]: mon.c calling monitor election 2024-09-10T18:57:26.043 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:25 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:57:26.043 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:25 smithi047 ceph-mon[29019]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:57:26.043 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:25 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:57:26.043 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:25 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:57:26.043 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:25 smithi047 ceph-mon[29019]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-10T18:57:26.043 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:25 smithi047 ceph-mon[29019]: monmap e3: 3 mons at {a=[v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0],b=[v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0],c=[v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-10T18:57:26.043 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:25 smithi047 ceph-mon[29019]: fsmap 2024-09-10T18:57:26.044 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:25 smithi047 ceph-mon[29019]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T18:57:26.044 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:25 smithi047 ceph-mon[29019]: mgrmap e13: y(active, since 47s) 2024-09-10T18:57:26.044 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:25 smithi047 ceph-mon[29019]: overall HEALTH_OK 2024-09-10T18:57:26.044 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:25 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:26.044 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:25 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:26.044 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:25 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:26.044 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:25 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:26.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:25 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:57:26.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:25 smithi149 ceph-mon[33230]: mon.a calling monitor election 2024-09-10T18:57:26.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:25 smithi149 ceph-mon[33230]: mon.b calling monitor election 2024-09-10T18:57:26.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:25 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:57:26.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:25 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:57:26.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:25 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:57:26.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:25 smithi149 ceph-mon[33230]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:57:26.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:25 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:57:26.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:25 smithi149 ceph-mon[33230]: mon.c calling monitor election 2024-09-10T18:57:26.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:25 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:57:26.094 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:25 smithi149 ceph-mon[33230]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:57:26.094 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:25 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:57:26.094 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:25 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:57:26.094 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:25 smithi149 ceph-mon[33230]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-10T18:57:26.094 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:25 smithi149 ceph-mon[33230]: monmap e3: 3 mons at {a=[v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0],b=[v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0],c=[v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-10T18:57:26.094 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:25 smithi149 ceph-mon[33230]: fsmap 2024-09-10T18:57:26.094 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:25 smithi149 ceph-mon[33230]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T18:57:26.094 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:25 smithi149 ceph-mon[33230]: mgrmap e13: y(active, since 47s) 2024-09-10T18:57:26.094 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:25 smithi149 ceph-mon[33230]: overall HEALTH_OK 2024-09-10T18:57:26.094 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:25 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:26.094 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:25 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:26.094 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:25 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:26.094 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:25 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:27.039 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:26 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:57:27.039 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:26 smithi047 ceph-mon[29019]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:57:27.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:26 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:57:27.040 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:26 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:57:27.040 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:26 smithi047 ceph-mon[39856]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:57:27.040 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:26 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:57:27.092 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:26 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:57:27.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:26 smithi149 ceph-mon[33230]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:57:27.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:26 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:57:27.461 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T18:57:27.462 INFO:teuthology.orchestra.run.smithi149.stdout:{"epoch":3,"fsid":"d8563d7e-6fa5-11ef-bcea-c7b262605968","modified":"2024-09-10T18:57:20.661875Z","created":"2024-09-10T18:55:41.695745Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:3300","nonce":0},{"type":"v1","addr":"172.21.15.149:6789","nonce":0}]},"addr":"172.21.15.149:6789/0","public_addr":"172.21.15.149:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3301","nonce":0},{"type":"v1","addr":"172.21.15.47:6790","nonce":0}]},"addr":"172.21.15.47:6790/0","public_addr":"172.21.15.47:6790/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-09-10T18:57:27.462 INFO:teuthology.orchestra.run.smithi149.stderr:dumped monmap epoch 3 2024-09-10T18:57:28.013 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:27 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:28.013 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:27 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:28.013 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:27 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:28.013 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:27 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:28.013 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:27 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:57:28.013 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:27 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:57:28.013 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:27 smithi149 ceph-mon[33230]: Updating smithi047:/etc/ceph/ceph.conf 2024-09-10T18:57:28.013 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:27 smithi149 ceph-mon[33230]: Updating smithi149:/etc/ceph/ceph.conf 2024-09-10T18:57:28.013 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:27 smithi149 ceph-mon[33230]: Updating smithi149:/var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:57:28.013 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:27 smithi149 ceph-mon[33230]: Updating smithi047:/var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:57:28.013 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:27 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:28.014 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:27 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:28.014 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:27 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:28.014 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:27 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:28.014 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:27 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:28.014 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:27 smithi149 ceph-mon[33230]: from='client.? 172.21.15.149:0/4145495965' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T18:57:28.014 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:27 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:57:28.014 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:27 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T18:57:28.014 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:27 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:57:28.036 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-10T18:57:28.036 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8563d7e-6fa5-11ef-bcea-c7b262605968 -- ceph config generate-minimal-conf 2024-09-10T18:57:28.039 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:27 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:28.040 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:27 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:28.040 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:27 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:28.040 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:27 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:28.040 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:27 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:57:28.040 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:27 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:57:28.040 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:27 smithi047 ceph-mon[39856]: Updating smithi047:/etc/ceph/ceph.conf 2024-09-10T18:57:28.040 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:27 smithi047 ceph-mon[39856]: Updating smithi149:/etc/ceph/ceph.conf 2024-09-10T18:57:28.040 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:27 smithi047 ceph-mon[39856]: Updating smithi149:/var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:57:28.040 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:27 smithi047 ceph-mon[39856]: Updating smithi047:/var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:57:28.040 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:27 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:28.040 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:27 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:28.040 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:27 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:28.040 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:27 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:28.041 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:27 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:28.041 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:27 smithi047 ceph-mon[39856]: from='client.? 172.21.15.149:0/4145495965' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T18:57:28.041 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:27 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:57:28.041 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:27 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T18:57:28.041 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:27 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:57:28.041 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:27 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:28.041 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:27 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:28.041 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:27 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:28.042 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:27 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:28.042 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:27 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:57:28.042 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:27 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:57:28.042 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:27 smithi047 ceph-mon[29019]: Updating smithi047:/etc/ceph/ceph.conf 2024-09-10T18:57:28.042 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:27 smithi047 ceph-mon[29019]: Updating smithi149:/etc/ceph/ceph.conf 2024-09-10T18:57:28.042 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:27 smithi047 ceph-mon[29019]: Updating smithi149:/var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:57:28.042 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:27 smithi047 ceph-mon[29019]: Updating smithi047:/var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:57:28.042 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:27 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:28.042 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:27 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:28.042 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:27 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:28.042 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:27 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:28.042 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:27 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:28.042 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:27 smithi047 ceph-mon[29019]: from='client.? 172.21.15.149:0/4145495965' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T18:57:28.042 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:27 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:57:28.043 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:27 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T18:57:28.043 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:27 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:57:29.024 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:57:29.145 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:28 smithi047 ceph-mon[39856]: Reconfiguring mon.a (unknown last config time)... 2024-09-10T18:57:29.146 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:28 smithi047 ceph-mon[39856]: Reconfiguring daemon mon.a on smithi047 2024-09-10T18:57:29.146 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:28 smithi047 ceph-mon[39856]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:57:29.146 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:28 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:29.146 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:28 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:29.146 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:28 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:29.146 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:28 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:57:29.146 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:28 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T18:57:29.146 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:28 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:57:29.146 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:28 smithi047 ceph-mon[29019]: Reconfiguring mon.a (unknown last config time)... 2024-09-10T18:57:29.147 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:28 smithi047 ceph-mon[29019]: Reconfiguring daemon mon.a on smithi047 2024-09-10T18:57:29.147 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:28 smithi047 ceph-mon[29019]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:57:29.147 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:28 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:29.147 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:28 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:29.147 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:28 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:29.147 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:28 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:57:29.147 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:28 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T18:57:29.147 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:28 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:57:29.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:28 smithi149 ceph-mon[33230]: Reconfiguring mon.a (unknown last config time)... 2024-09-10T18:57:29.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:28 smithi149 ceph-mon[33230]: Reconfiguring daemon mon.a on smithi047 2024-09-10T18:57:29.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:28 smithi149 ceph-mon[33230]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:57:29.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:28 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:29.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:28 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:29.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:28 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:29.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:28 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:57:29.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:28 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T18:57:29.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:28 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:57:30.232 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:29 smithi047 ceph-mon[39856]: Reconfiguring mon.c (monmap changed)... 2024-09-10T18:57:30.232 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:29 smithi047 ceph-mon[39856]: Reconfiguring daemon mon.c on smithi047 2024-09-10T18:57:30.232 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:29 smithi047 ceph-mon[29019]: Reconfiguring mon.c (monmap changed)... 2024-09-10T18:57:30.232 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:29 smithi047 ceph-mon[29019]: Reconfiguring daemon mon.c on smithi047 2024-09-10T18:57:30.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:29 smithi149 ceph-mon[33230]: Reconfiguring mon.c (monmap changed)... 2024-09-10T18:57:30.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:29 smithi149 ceph-mon[33230]: Reconfiguring daemon mon.c on smithi047 2024-09-10T18:57:31.227 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:30 smithi047 ceph-mon[29019]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:57:31.227 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:30 smithi047 ceph-mon[39856]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:57:31.342 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:30 smithi149 ceph-mon[33230]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:57:31.849 INFO:teuthology.orchestra.run.smithi047.stdout:# minimal ceph.conf for d8563d7e-6fa5-11ef-bcea-c7b262605968 2024-09-10T18:57:31.849 INFO:teuthology.orchestra.run.smithi047.stdout:[global] 2024-09-10T18:57:31.849 INFO:teuthology.orchestra.run.smithi047.stdout: fsid = d8563d7e-6fa5-11ef-bcea-c7b262605968 2024-09-10T18:57:31.849 INFO:teuthology.orchestra.run.smithi047.stdout: mon_host = [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-09-10T18:57:32.408 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:32 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:32.409 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:32 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:32.409 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:32 smithi047 ceph-mon[29019]: Reconfiguring mon.b (monmap changed)... 2024-09-10T18:57:32.409 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:32 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:57:32.409 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:32 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T18:57:32.409 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:32 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:57:32.409 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:32 smithi047 ceph-mon[29019]: Reconfiguring daemon mon.b on smithi149 2024-09-10T18:57:32.409 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:32 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/3068571094' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:57:32.409 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:32 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:32.409 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:32 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:32.409 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:32 smithi047 ceph-mon[39856]: Reconfiguring mon.b (monmap changed)... 2024-09-10T18:57:32.410 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:32 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:57:32.410 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:32 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T18:57:32.410 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:32 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:57:32.410 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:32 smithi047 ceph-mon[39856]: Reconfiguring daemon mon.b on smithi149 2024-09-10T18:57:32.410 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:32 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/3068571094' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:57:32.435 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-10T18:57:32.435 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-09-10T18:57:32.435 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/etc/ceph/ceph.conf 2024-09-10T18:57:32.505 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-09-10T18:57:32.505 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:57:32.573 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-09-10T18:57:32.573 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/etc/ceph/ceph.conf 2024-09-10T18:57:32.592 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:32 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:32.592 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:32 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:32.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:32 smithi149 ceph-mon[33230]: Reconfiguring mon.b (monmap changed)... 2024-09-10T18:57:32.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:32 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:57:32.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:32 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T18:57:32.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:32 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:57:32.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:32 smithi149 ceph-mon[33230]: Reconfiguring daemon mon.b on smithi149 2024-09-10T18:57:32.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:32 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/3068571094' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:57:32.602 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-09-10T18:57:32.602 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:57:32.668 INFO:tasks.cephadm:Adding mgr.y on smithi047 2024-09-10T18:57:32.669 INFO:tasks.cephadm:Adding mgr.x on smithi149 2024-09-10T18:57:32.669 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8563d7e-6fa5-11ef-bcea-c7b262605968 -- ceph orch apply mgr '2;smithi047=y;smithi149=x' 2024-09-10T18:57:32.942 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:57:33.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:33 smithi149 ceph-mon[33230]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:57:33.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:33 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:33.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:33 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:33.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:33 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:57:33.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:33 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:57:33.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:33 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:57:33.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:33 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:33.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:33 smithi047 ceph-mon[29019]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:57:33.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:33 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:33.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:33 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:33.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:33 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:57:33.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:33 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:57:33.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:33 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:57:33.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:33 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:33.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:33 smithi047 ceph-mon[39856]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:57:33.541 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:33 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:33.541 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:33 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:33.541 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:33 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:57:33.541 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:33 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:57:33.541 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:33 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:57:33.541 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:33 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:34.782 INFO:teuthology.orchestra.run.smithi149.stdout:Scheduled mgr update... 2024-09-10T18:57:35.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:35 smithi149 ceph-mon[33230]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:57:35.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:35 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:35.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:35 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:57:35.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:35 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:57:35.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:35 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:57:35.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:35 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:35.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:35 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T18:57:35.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:35 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T18:57:35.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:35 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T18:57:35.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:35 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:57:35.393 DEBUG:teuthology.orchestra.run.smithi149:mgr.x> sudo journalctl -f -n 0 -u ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968@mgr.x.service 2024-09-10T18:57:35.395 INFO:tasks.cephadm:Deploying OSDs... 2024-09-10T18:57:35.396 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-09-10T18:57:35.396 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T18:57:35.414 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-10T18:57:35.414 DEBUG:teuthology.orchestra.run.smithi047:> stat /dev/vg_nvme/lv_1 2024-09-10T18:57:35.472 INFO:teuthology.orchestra.run.smithi047.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-10T18:57:35.473 INFO:teuthology.orchestra.run.smithi047.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T18:57:35.473 INFO:teuthology.orchestra.run.smithi047.stdout:Device: 5h/5d Inode: 979 Links: 1 2024-09-10T18:57:35.473 INFO:teuthology.orchestra.run.smithi047.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T18:57:35.473 INFO:teuthology.orchestra.run.smithi047.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T18:57:35.473 INFO:teuthology.orchestra.run.smithi047.stdout:Access: 2024-09-10 18:57:20.041838851 +0000 2024-09-10T18:57:35.473 INFO:teuthology.orchestra.run.smithi047.stdout:Modify: 2024-09-10 18:50:26.288573385 +0000 2024-09-10T18:57:35.473 INFO:teuthology.orchestra.run.smithi047.stdout:Change: 2024-09-10 18:50:26.288573385 +0000 2024-09-10T18:57:35.473 INFO:teuthology.orchestra.run.smithi047.stdout: Birth: 2024-09-10 18:50:26.288573385 +0000 2024-09-10T18:57:35.473 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-10T18:57:35.542 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records in 2024-09-10T18:57:35.542 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records out 2024-09-10T18:57:35.542 INFO:teuthology.orchestra.run.smithi047.stderr:512 bytes copied, 0.000122572 s, 4.2 MB/s 2024-09-10T18:57:35.543 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:35 smithi047 ceph-mon[29019]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:57:35.543 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:35 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:35.543 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:35 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:57:35.543 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:35 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:57:35.543 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:35 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:57:35.543 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:35 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:35.543 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:35 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T18:57:35.543 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:35 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T18:57:35.543 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:35 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T18:57:35.543 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:35 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:57:35.544 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:35 smithi047 ceph-mon[39856]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:57:35.544 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:35 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:35.544 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:35 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:57:35.544 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:35 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:57:35.544 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:35 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:57:35.544 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:35 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:35.544 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:35 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T18:57:35.544 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:35 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T18:57:35.544 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:35 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T18:57:35.544 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:35 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:57:35.545 DEBUG:teuthology.orchestra.run.smithi047:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-10T18:57:35.603 DEBUG:teuthology.orchestra.run.smithi047:> stat /dev/vg_nvme/lv_2 2024-09-10T18:57:35.660 INFO:teuthology.orchestra.run.smithi047.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-10T18:57:35.660 INFO:teuthology.orchestra.run.smithi047.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T18:57:35.660 INFO:teuthology.orchestra.run.smithi047.stdout:Device: 5h/5d Inode: 974 Links: 1 2024-09-10T18:57:35.661 INFO:teuthology.orchestra.run.smithi047.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T18:57:35.661 INFO:teuthology.orchestra.run.smithi047.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T18:57:35.661 INFO:teuthology.orchestra.run.smithi047.stdout:Access: 2024-09-10 18:57:20.041838851 +0000 2024-09-10T18:57:35.661 INFO:teuthology.orchestra.run.smithi047.stdout:Modify: 2024-09-10 18:50:26.287573406 +0000 2024-09-10T18:57:35.661 INFO:teuthology.orchestra.run.smithi047.stdout:Change: 2024-09-10 18:50:26.287573406 +0000 2024-09-10T18:57:35.661 INFO:teuthology.orchestra.run.smithi047.stdout: Birth: 2024-09-10 18:50:26.287573406 +0000 2024-09-10T18:57:35.661 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-10T18:57:35.726 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records in 2024-09-10T18:57:35.726 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records out 2024-09-10T18:57:35.726 INFO:teuthology.orchestra.run.smithi047.stderr:512 bytes copied, 0.000110651 s, 4.6 MB/s 2024-09-10T18:57:35.727 DEBUG:teuthology.orchestra.run.smithi047:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-10T18:57:35.785 DEBUG:teuthology.orchestra.run.smithi047:> stat /dev/vg_nvme/lv_3 2024-09-10T18:57:35.841 INFO:teuthology.orchestra.run.smithi047.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-10T18:57:35.842 INFO:teuthology.orchestra.run.smithi047.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T18:57:35.842 INFO:teuthology.orchestra.run.smithi047.stdout:Device: 5h/5d Inode: 966 Links: 1 2024-09-10T18:57:35.842 INFO:teuthology.orchestra.run.smithi047.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T18:57:35.842 INFO:teuthology.orchestra.run.smithi047.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T18:57:35.842 INFO:teuthology.orchestra.run.smithi047.stdout:Access: 2024-09-10 18:57:20.042838830 +0000 2024-09-10T18:57:35.842 INFO:teuthology.orchestra.run.smithi047.stdout:Modify: 2024-09-10 18:50:26.287573406 +0000 2024-09-10T18:57:35.842 INFO:teuthology.orchestra.run.smithi047.stdout:Change: 2024-09-10 18:50:26.287573406 +0000 2024-09-10T18:57:35.842 INFO:teuthology.orchestra.run.smithi047.stdout: Birth: 2024-09-10 18:50:26.287573406 +0000 2024-09-10T18:57:35.842 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-10T18:57:35.907 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records in 2024-09-10T18:57:35.907 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records out 2024-09-10T18:57:35.908 INFO:teuthology.orchestra.run.smithi047.stderr:512 bytes copied, 0.000109584 s, 4.7 MB/s 2024-09-10T18:57:35.909 DEBUG:teuthology.orchestra.run.smithi047:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-10T18:57:35.967 DEBUG:teuthology.orchestra.run.smithi047:> stat /dev/vg_nvme/lv_4 2024-09-10T18:57:36.023 INFO:teuthology.orchestra.run.smithi047.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-10T18:57:36.024 INFO:teuthology.orchestra.run.smithi047.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T18:57:36.024 INFO:teuthology.orchestra.run.smithi047.stdout:Device: 5h/5d Inode: 981 Links: 1 2024-09-10T18:57:36.024 INFO:teuthology.orchestra.run.smithi047.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T18:57:36.024 INFO:teuthology.orchestra.run.smithi047.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T18:57:36.024 INFO:teuthology.orchestra.run.smithi047.stdout:Access: 2024-09-10 18:57:20.042838830 +0000 2024-09-10T18:57:36.024 INFO:teuthology.orchestra.run.smithi047.stdout:Modify: 2024-09-10 18:50:26.289573363 +0000 2024-09-10T18:57:36.024 INFO:teuthology.orchestra.run.smithi047.stdout:Change: 2024-09-10 18:50:26.289573363 +0000 2024-09-10T18:57:36.025 INFO:teuthology.orchestra.run.smithi047.stdout: Birth: 2024-09-10 18:50:26.289573363 +0000 2024-09-10T18:57:36.025 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-10T18:57:36.089 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records in 2024-09-10T18:57:36.089 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records out 2024-09-10T18:57:36.089 INFO:teuthology.orchestra.run.smithi047.stderr:512 bytes copied, 0.000108862 s, 4.7 MB/s 2024-09-10T18:57:36.090 DEBUG:teuthology.orchestra.run.smithi047:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-10T18:57:36.148 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-09-10T18:57:36.148 DEBUG:teuthology.orchestra.run.smithi149:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T18:57:36.166 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-10T18:57:36.166 DEBUG:teuthology.orchestra.run.smithi149:> stat /dev/vg_nvme/lv_1 2024-09-10T18:57:36.222 INFO:teuthology.orchestra.run.smithi149.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-10T18:57:36.222 INFO:teuthology.orchestra.run.smithi149.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T18:57:36.222 INFO:teuthology.orchestra.run.smithi149.stdout:Device: 5h/5d Inode: 1121 Links: 1 2024-09-10T18:57:36.223 INFO:teuthology.orchestra.run.smithi149.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T18:57:36.223 INFO:teuthology.orchestra.run.smithi149.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T18:57:36.223 INFO:teuthology.orchestra.run.smithi149.stdout:Access: 2024-09-10 18:57:11.791979843 +0000 2024-09-10T18:57:36.223 INFO:teuthology.orchestra.run.smithi149.stdout:Modify: 2024-09-10 18:50:26.417937206 +0000 2024-09-10T18:57:36.223 INFO:teuthology.orchestra.run.smithi149.stdout:Change: 2024-09-10 18:50:26.417937206 +0000 2024-09-10T18:57:36.223 INFO:teuthology.orchestra.run.smithi149.stdout: Birth: 2024-09-10 18:50:26.417937206 +0000 2024-09-10T18:57:36.223 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-10T18:57:36.288 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records in 2024-09-10T18:57:36.289 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records out 2024-09-10T18:57:36.289 INFO:teuthology.orchestra.run.smithi149.stderr:512 bytes copied, 0.000229478 s, 2.2 MB/s 2024-09-10T18:57:36.290 DEBUG:teuthology.orchestra.run.smithi149:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-10T18:57:36.347 DEBUG:teuthology.orchestra.run.smithi149:> stat /dev/vg_nvme/lv_2 2024-09-10T18:57:36.402 INFO:teuthology.orchestra.run.smithi149.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-10T18:57:36.402 INFO:teuthology.orchestra.run.smithi149.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T18:57:36.403 INFO:teuthology.orchestra.run.smithi149.stdout:Device: 5h/5d Inode: 1109 Links: 1 2024-09-10T18:57:36.403 INFO:teuthology.orchestra.run.smithi149.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T18:57:36.403 INFO:teuthology.orchestra.run.smithi149.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T18:57:36.403 INFO:teuthology.orchestra.run.smithi149.stdout:Access: 2024-09-10 18:57:11.791979843 +0000 2024-09-10T18:57:36.403 INFO:teuthology.orchestra.run.smithi149.stdout:Modify: 2024-09-10 18:50:26.417937206 +0000 2024-09-10T18:57:36.403 INFO:teuthology.orchestra.run.smithi149.stdout:Change: 2024-09-10 18:50:26.417937206 +0000 2024-09-10T18:57:36.403 INFO:teuthology.orchestra.run.smithi149.stdout: Birth: 2024-09-10 18:50:26.417937206 +0000 2024-09-10T18:57:36.403 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-10T18:57:36.465 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:36 smithi149 ceph-mon[33230]: from='client.24109 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi047=y;smithi149=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:57:36.465 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:36 smithi149 ceph-mon[33230]: Saving service mgr spec with placement smithi047=y;smithi149=x;count:2 2024-09-10T18:57:36.466 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:36 smithi149 ceph-mon[33230]: Deploying daemon mgr.x on smithi149 2024-09-10T18:57:36.555 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records in 2024-09-10T18:57:36.555 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records out 2024-09-10T18:57:36.555 INFO:teuthology.orchestra.run.smithi149.stderr:512 bytes copied, 0.000211208 s, 2.4 MB/s 2024-09-10T18:57:36.556 DEBUG:teuthology.orchestra.run.smithi149:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-10T18:57:36.619 DEBUG:teuthology.orchestra.run.smithi149:> stat /dev/vg_nvme/lv_3 2024-09-10T18:57:36.677 INFO:teuthology.orchestra.run.smithi149.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-10T18:57:36.677 INFO:teuthology.orchestra.run.smithi149.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T18:57:36.677 INFO:teuthology.orchestra.run.smithi149.stdout:Device: 5h/5d Inode: 1118 Links: 1 2024-09-10T18:57:36.677 INFO:teuthology.orchestra.run.smithi149.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T18:57:36.678 INFO:teuthology.orchestra.run.smithi149.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T18:57:36.678 INFO:teuthology.orchestra.run.smithi149.stdout:Access: 2024-09-10 18:57:36.508491908 +0000 2024-09-10T18:57:36.678 INFO:teuthology.orchestra.run.smithi149.stdout:Modify: 2024-09-10 18:50:26.417937206 +0000 2024-09-10T18:57:36.678 INFO:teuthology.orchestra.run.smithi149.stdout:Change: 2024-09-10 18:50:26.417937206 +0000 2024-09-10T18:57:36.678 INFO:teuthology.orchestra.run.smithi149.stdout: Birth: 2024-09-10 18:50:26.417937206 +0000 2024-09-10T18:57:36.678 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-10T18:57:36.762 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records in 2024-09-10T18:57:36.762 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records out 2024-09-10T18:57:36.762 INFO:teuthology.orchestra.run.smithi149.stderr:512 bytes copied, 0.000174868 s, 2.9 MB/s 2024-09-10T18:57:36.763 DEBUG:teuthology.orchestra.run.smithi149:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-10T18:57:36.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:36 smithi047 ceph-mon[29019]: from='client.24109 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi047=y;smithi149=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:57:36.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:36 smithi047 ceph-mon[29019]: Saving service mgr spec with placement smithi047=y;smithi149=x;count:2 2024-09-10T18:57:36.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:36 smithi047 ceph-mon[29019]: Deploying daemon mgr.x on smithi149 2024-09-10T18:57:36.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:36 smithi047 ceph-mon[39856]: from='client.24109 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi047=y;smithi149=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:57:36.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:36 smithi047 ceph-mon[39856]: Saving service mgr spec with placement smithi047=y;smithi149=x;count:2 2024-09-10T18:57:36.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:36 smithi047 ceph-mon[39856]: Deploying daemon mgr.x on smithi149 2024-09-10T18:57:36.820 DEBUG:teuthology.orchestra.run.smithi149:> stat /dev/vg_nvme/lv_4 2024-09-10T18:57:36.877 INFO:teuthology.orchestra.run.smithi149.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-10T18:57:36.877 INFO:teuthology.orchestra.run.smithi149.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T18:57:36.877 INFO:teuthology.orchestra.run.smithi149.stdout:Device: 5h/5d Inode: 1108 Links: 1 2024-09-10T18:57:36.877 INFO:teuthology.orchestra.run.smithi149.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T18:57:36.877 INFO:teuthology.orchestra.run.smithi149.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T18:57:36.878 INFO:teuthology.orchestra.run.smithi149.stdout:Access: 2024-09-10 18:57:36.689488335 +0000 2024-09-10T18:57:36.878 INFO:teuthology.orchestra.run.smithi149.stdout:Modify: 2024-09-10 18:50:26.417937206 +0000 2024-09-10T18:57:36.878 INFO:teuthology.orchestra.run.smithi149.stdout:Change: 2024-09-10 18:50:26.417937206 +0000 2024-09-10T18:57:36.878 INFO:teuthology.orchestra.run.smithi149.stdout: Birth: 2024-09-10 18:50:26.417937206 +0000 2024-09-10T18:57:36.878 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-10T18:57:36.941 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records in 2024-09-10T18:57:36.942 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records out 2024-09-10T18:57:36.942 INFO:teuthology.orchestra.run.smithi149.stderr:512 bytes copied, 0.0002143 s, 2.4 MB/s 2024-09-10T18:57:36.943 DEBUG:teuthology.orchestra.run.smithi149:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-10T18:57:36.999 INFO:tasks.cephadm:Deploying osd.0 on smithi047 with /dev/vg_nvme/lv_4... 2024-09-10T18:57:36.999 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8563d7e-6fa5-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-09-10T18:57:37.093 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 18:57:36 smithi149 podman[35022]: 2024-09-10 18:57:36.979320384 +0000 UTC m=+0.126010610 container create 4a344cd2abe1ccd59b148896058d6307a0b459b13e17726f8ee7e7d6d6e14525 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-10T18:57:37.259 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:57:37.412 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:37 smithi149 ceph-mon[33230]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:57:37.413 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 18:57:37 smithi149 podman[35022]: 2024-09-10 18:57:37.223351916 +0000 UTC m=+0.370042143 container init 4a344cd2abe1ccd59b148896058d6307a0b459b13e17726f8ee7e7d6d6e14525 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739) 2024-09-10T18:57:37.413 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 18:57:37 smithi149 podman[35022]: 2024-09-10 18:57:37.226034719 +0000 UTC m=+0.372724945 container start 4a344cd2abe1ccd59b148896058d6307a0b459b13e17726f8ee7e7d6d6e14525 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-10T18:57:37.413 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 18:57:37 smithi149 ceph-mgr[35094]: -- 172.21.15.149:0/3817493079 <== mon.1 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d425160340 con 0x55d42516d000 2024-09-10T18:57:37.413 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 18:57:37 smithi149 bash[35022]: 4a344cd2abe1ccd59b148896058d6307a0b459b13e17726f8ee7e7d6d6e14525 2024-09-10T18:57:37.413 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 18:57:37 smithi149 systemd[1]: Started Ceph mgr.x for d8563d7e-6fa5-11ef-bcea-c7b262605968. 2024-09-10T18:57:37.413 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 18:57:37 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T18:57:37.341+0000 7f0622055ec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T18:57:37.413 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 18:57:37 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T18:57:37.411+0000 7f0622055ec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T18:57:37.619 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:37 smithi047 ceph-mon[29019]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:57:37.620 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:37 smithi047 ceph-mon[39856]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:57:37.766 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 18:57:37 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T18:57:37.479+0000 7f0622055ec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T18:57:37.766 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 18:57:37 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T18:57:37.611+0000 7f0622055ec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T18:57:37.766 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 18:57:37 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T18:57:37.765+0000 7f0622055ec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T18:57:38.342 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 18:57:38 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T18:57:38.048+0000 7f0622055ec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T18:57:38.343 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 18:57:38 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T18:57:38.116+0000 7f0622055ec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T18:57:38.343 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 18:57:38 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T18:57:38.282+0000 7f0622055ec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T18:57:38.600 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:38 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:38.600 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:38 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:38.600 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:38 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:38.600 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:38 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:38.600 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:38 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:57:38.600 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:38 smithi047 ceph-mon[29019]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:57:38.600 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:38 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:38.601 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:38 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:38.601 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:38 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:38.601 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:38 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:38.601 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:38 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:38.601 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:38 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:57:38.601 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:38 smithi047 ceph-mon[39856]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:57:38.601 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:38 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:38.774 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:38 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:38.774 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:38 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:38.774 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:38 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:38.774 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:38 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:38.774 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:38 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:57:38.774 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:38 smithi149 ceph-mon[33230]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:57:38.774 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:38 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:38.965 INFO:teuthology.orchestra.run.smithi047.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-09-10T18:57:39.053 INFO:teuthology.orchestra.run.smithi047.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-09-10T18:57:39.064 INFO:teuthology.orchestra.run.smithi047.stderr: stderr: 10+0 records in 2024-09-10T18:57:39.064 INFO:teuthology.orchestra.run.smithi047.stderr:10+0 records out 2024-09-10T18:57:39.064 INFO:teuthology.orchestra.run.smithi047.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00896984 s, 1.2 GB/s 2024-09-10T18:57:39.064 INFO:teuthology.orchestra.run.smithi047.stderr:--> Zapping successful for: 2024-09-10T18:57:39.509 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 18:57:39 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T18:57:39.180+0000 7f0622055ec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T18:57:39.509 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 18:57:39 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T18:57:39.248+0000 7f0622055ec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T18:57:39.762 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:39 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:39.762 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:39 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:39.762 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:39 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:57:39.762 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:39 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:57:39.762 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:39 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:39.762 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:39 smithi149 ceph-mon[33230]: Reconfiguring mgr.y (unknown last config time)... 2024-09-10T18:57:39.762 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:39 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T18:57:39.762 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:39 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T18:57:39.763 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:39 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:57:39.763 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:39 smithi149 ceph-mon[33230]: Reconfiguring daemon mgr.y on smithi047 2024-09-10T18:57:39.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:39 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:39.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:39 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:39.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:39 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:57:39.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:39 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:57:39.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:39 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:39.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:39 smithi047 ceph-mon[29019]: Reconfiguring mgr.y (unknown last config time)... 2024-09-10T18:57:39.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:39 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T18:57:39.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:39 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T18:57:39.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:39 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:57:39.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:39 smithi047 ceph-mon[29019]: Reconfiguring daemon mgr.y on smithi047 2024-09-10T18:57:39.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:39 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:39.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:39 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:39.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:39 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:57:39.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:39 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:57:39.791 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:39 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:39.791 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:39 smithi047 ceph-mon[39856]: Reconfiguring mgr.y (unknown last config time)... 2024-09-10T18:57:39.791 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:39 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T18:57:39.791 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:39 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T18:57:39.791 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:39 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:57:39.791 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:39 smithi047 ceph-mon[39856]: Reconfiguring daemon mgr.y on smithi047 2024-09-10T18:57:40.092 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 18:57:39 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T18:57:39.985+0000 7f0622055ec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T18:57:40.152 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8563d7e-6fa5-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi047:vg_nvme/lv_4 2024-09-10T18:57:40.407 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 18:57:40 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T18:57:40.098+0000 7f0622055ec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T18:57:40.407 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 18:57:40 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T18:57:40.219+0000 7f0622055ec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T18:57:40.407 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 18:57:40 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T18:57:40.279+0000 7f0622055ec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T18:57:40.407 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 18:57:40 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T18:57:40.405+0000 7f0622055ec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T18:57:40.643 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:57:40.667 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:40 smithi149 ceph-mon[33230]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:57:40.668 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 18:57:40 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T18:57:40.472+0000 7f0622055ec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T18:57:40.668 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 18:57:40 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T18:57:40.530+0000 7f0622055ec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T18:57:40.668 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 18:57:40 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T18:57:40.666+0000 7f0622055ec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T18:57:40.764 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:40 smithi047 ceph-mon[29019]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:57:40.765 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:40 smithi047 ceph-mon[39856]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:57:40.934 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 18:57:40 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T18:57:40.731+0000 7f0622055ec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T18:57:40.935 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 18:57:40 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T18:57:40.796+0000 7f0622055ec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T18:57:40.935 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 18:57:40 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T18:57:40.875+0000 7f0622055ec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T18:57:40.935 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 18:57:40 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T18:57:40.933+0000 7f0622055ec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T18:57:41.342 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 18:57:40 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T18:57:40.991+0000 7f0622055ec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T18:57:41.788 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:41 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:41.789 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:41 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:41.789 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:41 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:57:41.789 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:41 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:57:41.789 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:41 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:57:41.789 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:41 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:41.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:41 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:41.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:41 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:41.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:41 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:57:41.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:41 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:57:41.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:41 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:57:41.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:41 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:42.092 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:41 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:42.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:41 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:42.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:41 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:57:42.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:41 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:57:42.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:41 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:57:42.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:41 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:42.093 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 18:57:41 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T18:57:41.739+0000 7f0622055ec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T18:57:42.890 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:42 smithi047 ceph-mon[29019]: Standby manager daemon x started 2024-09-10T18:57:42.890 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:42 smithi047 ceph-mon[29019]: from='mgr.? 172.21.15.149:0/1959972142' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-10T18:57:42.890 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:42 smithi047 ceph-mon[29019]: from='mgr.? 172.21.15.149:0/1959972142' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T18:57:42.890 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:42 smithi047 ceph-mon[29019]: from='mgr.? 172.21.15.149:0/1959972142' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-10T18:57:42.891 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:42 smithi047 ceph-mon[29019]: from='mgr.? 172.21.15.149:0/1959972142' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T18:57:42.891 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:42 smithi047 ceph-mon[29019]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:57:42.891 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:42 smithi047 ceph-mon[29019]: from='client.14217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:57:42.891 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:42 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:57:42.891 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:42 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:57:42.891 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:42 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:57:42.891 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:42 smithi047 ceph-mon[39856]: Standby manager daemon x started 2024-09-10T18:57:42.891 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:42 smithi047 ceph-mon[39856]: from='mgr.? 172.21.15.149:0/1959972142' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-10T18:57:42.891 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:42 smithi047 ceph-mon[39856]: from='mgr.? 172.21.15.149:0/1959972142' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T18:57:42.891 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:42 smithi047 ceph-mon[39856]: from='mgr.? 172.21.15.149:0/1959972142' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-10T18:57:42.892 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:42 smithi047 ceph-mon[39856]: from='mgr.? 172.21.15.149:0/1959972142' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T18:57:42.892 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:42 smithi047 ceph-mon[39856]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:57:42.892 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:42 smithi047 ceph-mon[39856]: from='client.14217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:57:42.892 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:42 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:57:42.892 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:42 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:57:42.892 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:42 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:57:43.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:42 smithi149 ceph-mon[33230]: Standby manager daemon x started 2024-09-10T18:57:43.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:42 smithi149 ceph-mon[33230]: from='mgr.? 172.21.15.149:0/1959972142' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-10T18:57:43.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:42 smithi149 ceph-mon[33230]: from='mgr.? 172.21.15.149:0/1959972142' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T18:57:43.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:42 smithi149 ceph-mon[33230]: from='mgr.? 172.21.15.149:0/1959972142' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-10T18:57:43.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:42 smithi149 ceph-mon[33230]: from='mgr.? 172.21.15.149:0/1959972142' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T18:57:43.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:42 smithi149 ceph-mon[33230]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:57:43.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:42 smithi149 ceph-mon[33230]: from='client.14217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:57:43.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:42 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:57:43.094 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:42 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:57:43.094 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:42 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:57:43.846 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:43 smithi047 ceph-mon[29019]: mgrmap e14: y(active, since 64s), standbys: x 2024-09-10T18:57:43.846 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:43 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-10T18:57:43.847 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:43 smithi047 ceph-mon[39856]: mgrmap e14: y(active, since 64s), standbys: x 2024-09-10T18:57:43.847 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:43 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-10T18:57:44.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:43 smithi149 ceph-mon[33230]: mgrmap e14: y(active, since 64s), standbys: x 2024-09-10T18:57:44.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:43 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-10T18:57:45.039 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:44 smithi047 ceph-mon[29019]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:57:45.039 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:44 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/2764067906' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b4576db6-5c73-4a0e-af6f-6aa568540249"}]: dispatch 2024-09-10T18:57:45.039 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:44 smithi047 ceph-mon[29019]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b4576db6-5c73-4a0e-af6f-6aa568540249"}]: dispatch 2024-09-10T18:57:45.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:44 smithi047 ceph-mon[29019]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b4576db6-5c73-4a0e-af6f-6aa568540249"}]': finished 2024-09-10T18:57:45.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:44 smithi047 ceph-mon[29019]: osdmap e5: 1 total, 0 up, 1 in 2024-09-10T18:57:45.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:44 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:57:45.040 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:44 smithi047 ceph-mon[39856]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:57:45.040 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:44 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/2764067906' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b4576db6-5c73-4a0e-af6f-6aa568540249"}]: dispatch 2024-09-10T18:57:45.040 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:44 smithi047 ceph-mon[39856]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b4576db6-5c73-4a0e-af6f-6aa568540249"}]: dispatch 2024-09-10T18:57:45.040 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:44 smithi047 ceph-mon[39856]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b4576db6-5c73-4a0e-af6f-6aa568540249"}]': finished 2024-09-10T18:57:45.040 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:44 smithi047 ceph-mon[39856]: osdmap e5: 1 total, 0 up, 1 in 2024-09-10T18:57:45.040 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:44 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:57:45.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:44 smithi149 ceph-mon[33230]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:57:45.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:44 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/2764067906' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b4576db6-5c73-4a0e-af6f-6aa568540249"}]: dispatch 2024-09-10T18:57:45.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:44 smithi149 ceph-mon[33230]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b4576db6-5c73-4a0e-af6f-6aa568540249"}]: dispatch 2024-09-10T18:57:45.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:44 smithi149 ceph-mon[33230]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b4576db6-5c73-4a0e-af6f-6aa568540249"}]': finished 2024-09-10T18:57:45.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:44 smithi149 ceph-mon[33230]: osdmap e5: 1 total, 0 up, 1 in 2024-09-10T18:57:45.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:44 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:57:46.039 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:45 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/1728261250' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:57:46.040 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:45 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/1728261250' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:57:46.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:45 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/1728261250' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:57:47.039 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:46 smithi047 ceph-mon[29019]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:57:47.040 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:46 smithi047 ceph-mon[39856]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:57:47.092 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:46 smithi149 ceph-mon[33230]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:57:49.256 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:49 smithi149 ceph-mon[33230]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:57:49.289 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:49 smithi047 ceph-mon[39856]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:57:49.289 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:49 smithi047 ceph-mon[29019]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:57:51.289 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:51 smithi047 ceph-mon[29019]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:57:51.289 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:51 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-10T18:57:51.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:51 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:57:51.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:51 smithi047 ceph-mon[29019]: Deploying daemon osd.0 on smithi047 2024-09-10T18:57:51.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:51 smithi047 ceph-mon[39856]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:57:51.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:51 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-10T18:57:51.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:51 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:57:51.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:51 smithi047 ceph-mon[39856]: Deploying daemon osd.0 on smithi047 2024-09-10T18:57:51.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:51 smithi149 ceph-mon[33230]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:57:51.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:51 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-10T18:57:51.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:51 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:57:51.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:51 smithi149 ceph-mon[33230]: Deploying daemon osd.0 on smithi047 2024-09-10T18:57:53.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:53 smithi047 ceph-mon[29019]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:57:53.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:53 smithi047 ceph-mon[39856]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:57:53.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:53 smithi149 ceph-mon[33230]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:57:55.289 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:55 smithi047 ceph-mon[29019]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:57:55.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:55 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:57:55.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:55 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:55.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:55 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:55.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:55 smithi047 ceph-mon[39856]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:57:55.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:55 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:57:55.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:55 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:55.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:55 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:55.342 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:55 smithi149 ceph-mon[33230]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:57:55.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:55 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:57:55.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:55 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:55.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:55 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:57.289 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:57 smithi047 ceph-mon[29019]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:57:57.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:57 smithi047 ceph-mon[39856]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:57:57.342 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:57 smithi149 ceph-mon[33230]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:57:58.193 INFO:teuthology.orchestra.run.smithi047.stdout:Created osd(s) 0 on host 'smithi047' 2024-09-10T18:57:58.726 DEBUG:teuthology.orchestra.run.smithi047:osd.0> sudo journalctl -f -n 0 -u ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968@osd.0.service 2024-09-10T18:57:58.728 INFO:tasks.cephadm:Deploying osd.1 on smithi047 with /dev/vg_nvme/lv_3... 2024-09-10T18:57:58.729 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8563d7e-6fa5-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-09-10T18:57:58.749 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:58 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:58.749 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:58 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:58.749 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:58 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:57:58.750 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:58 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:57:58.750 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:58 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:58.750 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:58 smithi047 ceph-mon[39856]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:57:58.750 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:58 smithi047 ceph-mon[39856]: from='osd.0 v2:172.21.15.47:6801/840905999' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-10T18:57:58.750 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:58 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:57:58.750 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:58 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:58.750 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:58 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:58.751 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:58 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:58.751 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:58 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:58.751 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:58 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:57:58.751 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:58 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:57:58.751 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:58 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:58.752 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:58 smithi047 ceph-mon[29019]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:57:58.752 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:58 smithi047 ceph-mon[29019]: from='osd.0 v2:172.21.15.47:6801/840905999' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-10T18:57:58.752 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:58 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:57:58.752 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:58 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:58.752 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:58 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:58.843 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:58 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:58.843 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:58 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:58.843 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:58 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:57:58.843 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:58 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:57:58.843 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:58 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:58.843 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:58 smithi149 ceph-mon[33230]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:57:58.843 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:58 smithi149 ceph-mon[33230]: from='osd.0 v2:172.21.15.47:6801/840905999' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-10T18:57:58.843 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:58 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:57:58.843 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:58 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:58.843 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:58 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:57:59.789 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:59 smithi047 ceph-mon[39856]: from='osd.0 v2:172.21.15.47:6801/840905999' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-10T18:57:59.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:59 smithi047 ceph-mon[39856]: osdmap e6: 1 total, 0 up, 1 in 2024-09-10T18:57:59.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:59 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:57:59.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:57:59 smithi047 ceph-mon[39856]: from='osd.0 v2:172.21.15.47:6801/840905999' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-09-10T18:57:59.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:59 smithi047 ceph-mon[29019]: from='osd.0 v2:172.21.15.47:6801/840905999' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-10T18:57:59.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:59 smithi047 ceph-mon[29019]: osdmap e6: 1 total, 0 up, 1 in 2024-09-10T18:57:59.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:59 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:57:59.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:57:59 smithi047 ceph-mon[29019]: from='osd.0 v2:172.21.15.47:6801/840905999' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-09-10T18:57:59.790 INFO:journalctl@ceph.osd.0.smithi047.stdout:Sep 10 18:57:59 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-0[43856]: 2024-09-10T18:57:59.534+0000 7fb93929f640 -1 osd.0 0 waiting for initial osdmap 2024-09-10T18:57:59.791 INFO:journalctl@ceph.osd.0.smithi047.stdout:Sep 10 18:57:59 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-0[43856]: 2024-09-10T18:57:59.536+0000 7fb9308ad640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T18:57:59.842 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:59 smithi149 ceph-mon[33230]: from='osd.0 v2:172.21.15.47:6801/840905999' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-10T18:57:59.843 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:59 smithi149 ceph-mon[33230]: osdmap e6: 1 total, 0 up, 1 in 2024-09-10T18:57:59.843 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:59 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:57:59.843 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:57:59 smithi149 ceph-mon[33230]: from='osd.0 v2:172.21.15.47:6801/840905999' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-09-10T18:57:59.875 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:58:00.675 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:00 smithi047 ceph-mon[39856]: from='osd.0 v2:172.21.15.47:6801/840905999' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-09-10T18:58:00.675 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:00 smithi047 ceph-mon[39856]: osdmap e7: 1 total, 0 up, 1 in 2024-09-10T18:58:00.676 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:00 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:58:00.676 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:00 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:58:00.676 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:00 smithi047 ceph-mon[39856]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:58:00.676 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:00 smithi047 ceph-mon[29019]: from='osd.0 v2:172.21.15.47:6801/840905999' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-09-10T18:58:00.676 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:00 smithi047 ceph-mon[29019]: osdmap e7: 1 total, 0 up, 1 in 2024-09-10T18:58:00.676 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:00 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:58:00.676 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:00 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:58:00.676 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:00 smithi047 ceph-mon[29019]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:58:00.843 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:00 smithi149 ceph-mon[33230]: from='osd.0 v2:172.21.15.47:6801/840905999' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-09-10T18:58:00.843 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:00 smithi149 ceph-mon[33230]: osdmap e7: 1 total, 0 up, 1 in 2024-09-10T18:58:00.843 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:00 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:58:00.843 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:00 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:58:00.843 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:00 smithi149 ceph-mon[33230]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:58:01.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:01 smithi047 ceph-mon[29019]: purged_snaps scrub starts 2024-09-10T18:58:01.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:01 smithi047 ceph-mon[29019]: purged_snaps scrub ok 2024-09-10T18:58:01.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:01 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:58:01.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:01 smithi047 ceph-mon[29019]: osd.0 v2:172.21.15.47:6801/840905999 boot 2024-09-10T18:58:01.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:01 smithi047 ceph-mon[29019]: osdmap e8: 1 total, 1 up, 1 in 2024-09-10T18:58:01.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:01 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:58:01.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:01 smithi047 ceph-mon[39856]: purged_snaps scrub starts 2024-09-10T18:58:01.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:01 smithi047 ceph-mon[39856]: purged_snaps scrub ok 2024-09-10T18:58:01.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:01 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:58:01.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:01 smithi047 ceph-mon[39856]: osd.0 v2:172.21.15.47:6801/840905999 boot 2024-09-10T18:58:01.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:01 smithi047 ceph-mon[39856]: osdmap e8: 1 total, 1 up, 1 in 2024-09-10T18:58:01.791 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:01 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:58:01.844 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:01 smithi149 ceph-mon[33230]: purged_snaps scrub starts 2024-09-10T18:58:01.844 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:01 smithi149 ceph-mon[33230]: purged_snaps scrub ok 2024-09-10T18:58:01.844 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:01 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:58:01.844 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:01 smithi149 ceph-mon[33230]: osd.0 v2:172.21.15.47:6801/840905999 boot 2024-09-10T18:58:01.844 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:01 smithi149 ceph-mon[33230]: osdmap e8: 1 total, 1 up, 1 in 2024-09-10T18:58:01.844 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:01 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:58:01.847 INFO:teuthology.orchestra.run.smithi047.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-09-10T18:58:01.933 INFO:teuthology.orchestra.run.smithi047.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-09-10T18:58:01.945 INFO:teuthology.orchestra.run.smithi047.stderr: stderr: 10+0 records in 2024-09-10T18:58:01.945 INFO:teuthology.orchestra.run.smithi047.stderr:10+0 records out 2024-09-10T18:58:01.945 INFO:teuthology.orchestra.run.smithi047.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00991236 s, 1.1 GB/s 2024-09-10T18:58:01.945 INFO:teuthology.orchestra.run.smithi047.stderr:--> Zapping successful for: 2024-09-10T18:58:02.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:02 smithi047 ceph-mon[29019]: osdmap e9: 1 total, 1 up, 1 in 2024-09-10T18:58:02.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:02 smithi047 ceph-mon[29019]: pgmap v41: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:58:02.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:02 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:58:02.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:02 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:58:02.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:02 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:58:02.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:02 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:58:02.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:02 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:58:02.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:02 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:58:02.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:02 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:58:02.712 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:02 smithi047 ceph-mon[39856]: osdmap e9: 1 total, 1 up, 1 in 2024-09-10T18:58:02.713 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:02 smithi047 ceph-mon[39856]: pgmap v41: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:58:02.713 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:02 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:58:02.713 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:02 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:58:02.713 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:02 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:58:02.713 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:02 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:58:02.713 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:02 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:58:02.713 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:02 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:58:02.713 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:02 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:58:02.843 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:02 smithi149 ceph-mon[33230]: osdmap e9: 1 total, 1 up, 1 in 2024-09-10T18:58:02.843 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:02 smithi149 ceph-mon[33230]: pgmap v41: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:58:02.843 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:02 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:58:02.843 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:02 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:58:02.843 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:02 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:58:02.843 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:02 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:58:02.843 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:02 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:58:02.843 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:02 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:58:02.843 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:02 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:58:03.006 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8563d7e-6fa5-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi047:vg_nvme/lv_3 2024-09-10T18:58:03.257 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:58:03.843 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:03 smithi149 ceph-mon[33230]: Detected new or changed devices on smithi047 2024-09-10T18:58:03.843 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:03 smithi149 ceph-mon[33230]: Adjusting osd_memory_target on smithi047 to 16068M 2024-09-10T18:58:04.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:03 smithi047 ceph-mon[29019]: Detected new or changed devices on smithi047 2024-09-10T18:58:04.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:03 smithi047 ceph-mon[29019]: Adjusting osd_memory_target on smithi047 to 16068M 2024-09-10T18:58:04.040 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:03 smithi047 ceph-mon[39856]: Detected new or changed devices on smithi047 2024-09-10T18:58:04.040 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:03 smithi047 ceph-mon[39856]: Adjusting osd_memory_target on smithi047 to 16068M 2024-09-10T18:58:04.788 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:04 smithi047 ceph-mon[39856]: pgmap v42: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:58:04.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:04 smithi047 ceph-mon[29019]: pgmap v42: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:58:04.842 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:04 smithi149 ceph-mon[33230]: pgmap v42: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:58:05.657 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:05 smithi047 ceph-mon[29019]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:58:05.658 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:05 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:58:05.658 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:05 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:58:05.658 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:05 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:58:05.658 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:05 smithi047 ceph-mon[39856]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:58:05.658 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:05 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:58:05.658 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:05 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:58:05.658 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:05 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:58:05.842 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:05 smithi149 ceph-mon[33230]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:58:05.843 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:05 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:58:05.843 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:05 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:58:05.843 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:05 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:58:06.677 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:06 smithi047 ceph-mon[29019]: pgmap v43: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:58:06.678 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:06 smithi047 ceph-mon[39856]: pgmap v43: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:58:06.843 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:06 smithi149 ceph-mon[33230]: pgmap v43: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:58:07.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:07 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/1106746378' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5ecc9667-1ec5-47d0-b8b0-53411fa13b0b"}]: dispatch 2024-09-10T18:58:07.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:07 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/1106746378' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5ecc9667-1ec5-47d0-b8b0-53411fa13b0b"}]': finished 2024-09-10T18:58:07.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:07 smithi047 ceph-mon[29019]: osdmap e10: 2 total, 1 up, 2 in 2024-09-10T18:58:07.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:07 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:58:07.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:07 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/1106746378' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5ecc9667-1ec5-47d0-b8b0-53411fa13b0b"}]: dispatch 2024-09-10T18:58:07.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:07 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/1106746378' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5ecc9667-1ec5-47d0-b8b0-53411fa13b0b"}]': finished 2024-09-10T18:58:07.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:07 smithi047 ceph-mon[39856]: osdmap e10: 2 total, 1 up, 2 in 2024-09-10T18:58:07.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:07 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:58:07.842 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:07 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/1106746378' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5ecc9667-1ec5-47d0-b8b0-53411fa13b0b"}]: dispatch 2024-09-10T18:58:07.843 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:07 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/1106746378' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5ecc9667-1ec5-47d0-b8b0-53411fa13b0b"}]': finished 2024-09-10T18:58:07.843 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:07 smithi149 ceph-mon[33230]: osdmap e10: 2 total, 1 up, 2 in 2024-09-10T18:58:07.843 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:07 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:58:08.843 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:08 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/1248614391' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:58:08.843 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:08 smithi149 ceph-mon[33230]: pgmap v45: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:58:09.039 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:08 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/1248614391' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:58:09.039 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:08 smithi047 ceph-mon[29019]: pgmap v45: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:58:09.040 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:08 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/1248614391' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:58:09.040 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:08 smithi047 ceph-mon[39856]: pgmap v45: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:58:11.289 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:11 smithi047 ceph-mon[39856]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:58:11.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:11 smithi047 ceph-mon[29019]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:58:11.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:11 smithi149 ceph-mon[33230]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:58:13.097 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:13 smithi047 ceph-mon[29019]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:58:13.098 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:13 smithi047 ceph-mon[39856]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:58:13.342 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:13 smithi149 ceph-mon[33230]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:58:14.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:14 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-10T18:58:14.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:14 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:58:14.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:14 smithi047 ceph-mon[29019]: Deploying daemon osd.1 on smithi047 2024-09-10T18:58:14.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:14 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-10T18:58:14.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:14 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:58:14.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:14 smithi047 ceph-mon[39856]: Deploying daemon osd.1 on smithi047 2024-09-10T18:58:14.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:14 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-10T18:58:14.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:14 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:58:14.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:14 smithi149 ceph-mon[33230]: Deploying daemon osd.1 on smithi047 2024-09-10T18:58:15.289 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:15 smithi047 ceph-mon[29019]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:58:15.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:15 smithi047 ceph-mon[39856]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:58:15.342 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:15 smithi149 ceph-mon[33230]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:58:17.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:17 smithi047 ceph-mon[29019]: pgmap v49: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:58:17.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:17 smithi047 ceph-mon[39856]: pgmap v49: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:58:17.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:17 smithi149 ceph-mon[33230]: pgmap v49: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:58:18.289 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:18 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:58:18.289 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:18 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:58:18.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:18 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:58:18.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:18 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:58:18.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:18 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:58:18.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:18 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:58:18.342 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:18 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:58:18.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:18 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:58:18.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:18 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:58:19.018 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:19 smithi047 ceph-mon[29019]: pgmap v50: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:58:19.019 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:19 smithi047 ceph-mon[39856]: pgmap v50: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:58:19.342 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:19 smithi149 ceph-mon[33230]: pgmap v50: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:58:20.629 INFO:teuthology.orchestra.run.smithi047.stdout:Created osd(s) 1 on host 'smithi047' 2024-09-10T18:58:21.281 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:21 smithi047 ceph-mon[29019]: pgmap v51: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:58:21.281 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:21 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:58:21.281 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:21 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:58:21.281 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:21 smithi047 ceph-mon[29019]: from='osd.1 v2:172.21.15.47:6805/3075184801' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-10T18:58:21.282 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:21 smithi047 ceph-mon[39856]: pgmap v51: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:58:21.282 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:21 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:58:21.282 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:21 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:58:21.282 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:21 smithi047 ceph-mon[39856]: from='osd.1 v2:172.21.15.47:6805/3075184801' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-10T18:58:21.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:21 smithi149 ceph-mon[33230]: pgmap v51: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:58:21.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:21 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:58:21.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:21 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:58:21.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:21 smithi149 ceph-mon[33230]: from='osd.1 v2:172.21.15.47:6805/3075184801' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-10T18:58:21.598 DEBUG:teuthology.orchestra.run.smithi047:osd.1> sudo journalctl -f -n 0 -u ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968@osd.1.service 2024-09-10T18:58:21.600 INFO:tasks.cephadm:Deploying osd.2 on smithi047 with /dev/vg_nvme/lv_2... 2024-09-10T18:58:21.600 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8563d7e-6fa5-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-09-10T18:58:21.912 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:58:22.897 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:22 smithi047 ceph-mon[29019]: from='osd.1 v2:172.21.15.47:6805/3075184801' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-10T18:58:22.897 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:22 smithi047 ceph-mon[29019]: osdmap e11: 2 total, 1 up, 2 in 2024-09-10T18:58:22.897 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:22 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:58:22.897 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:22 smithi047 ceph-mon[29019]: from='osd.1 v2:172.21.15.47:6805/3075184801' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-09-10T18:58:22.897 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:22 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:58:22.897 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:22 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:58:22.897 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:22 smithi047 ceph-mon[29019]: pgmap v53: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:58:22.898 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:22 smithi047 ceph-mon[39856]: from='osd.1 v2:172.21.15.47:6805/3075184801' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-10T18:58:22.898 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:22 smithi047 ceph-mon[39856]: osdmap e11: 2 total, 1 up, 2 in 2024-09-10T18:58:22.898 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:22 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:58:22.898 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:22 smithi047 ceph-mon[39856]: from='osd.1 v2:172.21.15.47:6805/3075184801' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-09-10T18:58:22.898 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:22 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:58:22.898 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:22 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:58:22.898 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:22 smithi047 ceph-mon[39856]: pgmap v53: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:58:22.898 INFO:journalctl@ceph.osd.1.smithi047.stdout:Sep 10 18:58:22 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-1[49392]: 2024-09-10T18:58:22.632+0000 7f4b0965d640 -1 osd.1 0 waiting for initial osdmap 2024-09-10T18:58:22.898 INFO:journalctl@ceph.osd.1.smithi047.stdout:Sep 10 18:58:22 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-1[49392]: 2024-09-10T18:58:22.638+0000 7f4b02490640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T18:58:23.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:22 smithi149 ceph-mon[33230]: from='osd.1 v2:172.21.15.47:6805/3075184801' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-10T18:58:23.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:22 smithi149 ceph-mon[33230]: osdmap e11: 2 total, 1 up, 2 in 2024-09-10T18:58:23.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:22 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:58:23.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:22 smithi149 ceph-mon[33230]: from='osd.1 v2:172.21.15.47:6805/3075184801' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-09-10T18:58:23.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:22 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:58:23.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:22 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:58:23.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:22 smithi149 ceph-mon[33230]: pgmap v53: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:58:23.495 INFO:teuthology.orchestra.run.smithi047.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-09-10T18:58:23.579 INFO:teuthology.orchestra.run.smithi047.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-09-10T18:58:23.592 INFO:teuthology.orchestra.run.smithi047.stderr: stderr: 10+0 records in 2024-09-10T18:58:23.592 INFO:teuthology.orchestra.run.smithi047.stderr:10+0 records out 2024-09-10T18:58:23.592 INFO:teuthology.orchestra.run.smithi047.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00995025 s, 1.1 GB/s 2024-09-10T18:58:23.592 INFO:teuthology.orchestra.run.smithi047.stderr:--> Zapping successful for: 2024-09-10T18:58:23.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:23 smithi047 ceph-mon[29019]: from='osd.1 v2:172.21.15.47:6805/3075184801' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-09-10T18:58:23.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:23 smithi047 ceph-mon[29019]: osdmap e12: 2 total, 1 up, 2 in 2024-09-10T18:58:23.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:23 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:58:23.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:23 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:58:23.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:23 smithi047 ceph-mon[39856]: from='osd.1 v2:172.21.15.47:6805/3075184801' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-09-10T18:58:23.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:23 smithi047 ceph-mon[39856]: osdmap e12: 2 total, 1 up, 2 in 2024-09-10T18:58:23.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:23 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:58:23.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:23 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:58:24.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:23 smithi149 ceph-mon[33230]: from='osd.1 v2:172.21.15.47:6805/3075184801' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-09-10T18:58:24.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:23 smithi149 ceph-mon[33230]: osdmap e12: 2 total, 1 up, 2 in 2024-09-10T18:58:24.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:23 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:58:24.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:23 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:58:24.836 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:24 smithi047 ceph-mon[29019]: purged_snaps scrub starts 2024-09-10T18:58:24.836 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:24 smithi047 ceph-mon[29019]: purged_snaps scrub ok 2024-09-10T18:58:24.836 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:24 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:58:24.836 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:24 smithi047 ceph-mon[29019]: osd.1 v2:172.21.15.47:6805/3075184801 boot 2024-09-10T18:58:24.836 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:24 smithi047 ceph-mon[29019]: osdmap e13: 2 total, 2 up, 2 in 2024-09-10T18:58:24.836 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:24 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:58:24.836 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:24 smithi047 ceph-mon[29019]: pgmap v56: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:58:24.836 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:24 smithi047 ceph-mon[39856]: purged_snaps scrub starts 2024-09-10T18:58:24.837 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:24 smithi047 ceph-mon[39856]: purged_snaps scrub ok 2024-09-10T18:58:24.837 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:24 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:58:24.837 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:24 smithi047 ceph-mon[39856]: osd.1 v2:172.21.15.47:6805/3075184801 boot 2024-09-10T18:58:24.837 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:24 smithi047 ceph-mon[39856]: osdmap e13: 2 total, 2 up, 2 in 2024-09-10T18:58:24.837 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:24 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:58:24.837 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:24 smithi047 ceph-mon[39856]: pgmap v56: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:58:24.863 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8563d7e-6fa5-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi047:vg_nvme/lv_2 2024-09-10T18:58:25.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:24 smithi149 ceph-mon[33230]: purged_snaps scrub starts 2024-09-10T18:58:25.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:24 smithi149 ceph-mon[33230]: purged_snaps scrub ok 2024-09-10T18:58:25.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:24 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:58:25.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:24 smithi149 ceph-mon[33230]: osd.1 v2:172.21.15.47:6805/3075184801 boot 2024-09-10T18:58:25.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:24 smithi149 ceph-mon[33230]: osdmap e13: 2 total, 2 up, 2 in 2024-09-10T18:58:25.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:24 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:58:25.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:24 smithi149 ceph-mon[33230]: pgmap v56: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:58:25.388 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:58:25.764 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:25 smithi047 ceph-mon[29019]: osdmap e14: 2 total, 2 up, 2 in 2024-09-10T18:58:25.764 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:25 smithi047 ceph-mon[39856]: osdmap e14: 2 total, 2 up, 2 in 2024-09-10T18:58:26.092 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:25 smithi149 ceph-mon[33230]: osdmap e14: 2 total, 2 up, 2 in 2024-09-10T18:58:27.038 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:26 smithi047 ceph-mon[29019]: pgmap v58: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:58:27.038 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:26 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:58:27.038 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:26 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:58:27.038 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:26 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:58:27.038 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:26 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:58:27.039 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:26 smithi047 ceph-mon[39856]: pgmap v58: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:58:27.039 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:26 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:58:27.039 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:26 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:58:27.039 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:26 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:58:27.039 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:26 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:58:27.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:26 smithi149 ceph-mon[33230]: pgmap v58: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:58:27.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:26 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:58:27.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:26 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:58:27.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:26 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:58:27.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:26 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:58:28.039 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:27 smithi047 ceph-mon[29019]: Detected new or changed devices on smithi047 2024-09-10T18:58:28.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:27 smithi047 ceph-mon[29019]: Adjusting osd_memory_target on smithi047 to 8034M 2024-09-10T18:58:28.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:27 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:58:28.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:27 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:58:28.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:27 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:58:28.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:27 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:58:28.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:27 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:58:28.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:27 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:58:28.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:27 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:58:28.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:27 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:58:28.041 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:27 smithi047 ceph-mon[39856]: Detected new or changed devices on smithi047 2024-09-10T18:58:28.041 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:27 smithi047 ceph-mon[39856]: Adjusting osd_memory_target on smithi047 to 8034M 2024-09-10T18:58:28.041 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:27 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:58:28.041 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:27 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:58:28.041 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:27 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:58:28.041 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:27 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:58:28.041 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:27 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:58:28.041 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:27 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:58:28.041 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:27 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:58:28.041 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:27 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:58:28.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:27 smithi149 ceph-mon[33230]: Detected new or changed devices on smithi047 2024-09-10T18:58:28.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:27 smithi149 ceph-mon[33230]: Adjusting osd_memory_target on smithi047 to 8034M 2024-09-10T18:58:28.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:27 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:58:28.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:27 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:58:28.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:27 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:58:28.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:27 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:58:28.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:27 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:58:28.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:27 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:58:28.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:27 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:58:28.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:27 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:58:28.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:28 smithi047 ceph-mon[29019]: from='client.14271 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:58:28.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:28 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:58:28.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:28 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:58:28.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:28 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:58:28.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:28 smithi047 ceph-mon[29019]: pgmap v59: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:58:28.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:28 smithi047 ceph-mon[39856]: from='client.14271 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:58:28.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:28 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:58:28.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:28 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:58:28.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:28 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:58:28.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:28 smithi047 ceph-mon[39856]: pgmap v59: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:58:29.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:28 smithi149 ceph-mon[33230]: from='client.14271 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:58:29.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:28 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:58:29.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:28 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:58:29.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:28 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:58:29.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:28 smithi149 ceph-mon[33230]: pgmap v59: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:58:30.289 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:30 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/687051948' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "68a8c2ac-958c-411c-8a8f-f9e0b453901d"}]: dispatch 2024-09-10T18:58:30.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:30 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/687051948' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "68a8c2ac-958c-411c-8a8f-f9e0b453901d"}]': finished 2024-09-10T18:58:30.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:30 smithi047 ceph-mon[39856]: osdmap e15: 3 total, 2 up, 3 in 2024-09-10T18:58:30.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:30 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:58:30.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:30 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/687051948' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "68a8c2ac-958c-411c-8a8f-f9e0b453901d"}]: dispatch 2024-09-10T18:58:30.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:30 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/687051948' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "68a8c2ac-958c-411c-8a8f-f9e0b453901d"}]': finished 2024-09-10T18:58:30.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:30 smithi047 ceph-mon[29019]: osdmap e15: 3 total, 2 up, 3 in 2024-09-10T18:58:30.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:30 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:58:30.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:30 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/687051948' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "68a8c2ac-958c-411c-8a8f-f9e0b453901d"}]: dispatch 2024-09-10T18:58:30.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:30 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/687051948' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "68a8c2ac-958c-411c-8a8f-f9e0b453901d"}]': finished 2024-09-10T18:58:30.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:30 smithi149 ceph-mon[33230]: osdmap e15: 3 total, 2 up, 3 in 2024-09-10T18:58:30.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:30 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:58:31.289 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:31 smithi047 ceph-mon[29019]: pgmap v60: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:58:31.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:31 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/1106125878' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:58:31.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:31 smithi047 ceph-mon[39856]: pgmap v60: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:58:31.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:31 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/1106125878' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:58:31.342 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:31 smithi149 ceph-mon[33230]: pgmap v60: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:58:31.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:31 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/1106125878' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:58:33.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:33 smithi047 ceph-mon[29019]: pgmap v62: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:58:33.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:33 smithi047 ceph-mon[39856]: pgmap v62: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:58:33.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:33 smithi149 ceph-mon[33230]: pgmap v62: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:58:35.210 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:35 smithi047 ceph-mon[29019]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:58:35.211 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:35 smithi047 ceph-mon[39856]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:58:35.342 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:35 smithi149 ceph-mon[33230]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:58:36.247 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:36 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-10T18:58:36.247 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:36 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:58:36.247 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:36 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-10T18:58:36.247 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:36 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:58:36.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:36 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-10T18:58:36.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:36 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:58:37.289 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:37 smithi047 ceph-mon[29019]: Deploying daemon osd.2 on smithi047 2024-09-10T18:58:37.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:37 smithi047 ceph-mon[29019]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:58:37.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:37 smithi047 ceph-mon[39856]: Deploying daemon osd.2 on smithi047 2024-09-10T18:58:37.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:37 smithi047 ceph-mon[39856]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:58:37.342 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:37 smithi149 ceph-mon[33230]: Deploying daemon osd.2 on smithi047 2024-09-10T18:58:37.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:37 smithi149 ceph-mon[33230]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:58:39.264 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:39 smithi047 ceph-mon[29019]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:58:39.264 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:39 smithi047 ceph-mon[39856]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:58:39.342 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:39 smithi149 ceph-mon[33230]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:58:41.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:41 smithi047 ceph-mon[29019]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:58:41.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:41 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:58:41.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:41 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:58:41.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:41 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:58:41.291 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:41 smithi047 ceph-mon[39856]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:58:41.291 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:41 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:58:41.291 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:41 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:58:41.291 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:41 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:58:41.342 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:41 smithi149 ceph-mon[33230]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:58:41.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:41 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:58:41.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:41 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:58:41.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:41 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:58:43.122 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:43 smithi047 ceph-mon[29019]: pgmap v67: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:58:43.123 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:43 smithi047 ceph-mon[39856]: pgmap v67: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:58:43.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:43 smithi149 ceph-mon[33230]: pgmap v67: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:58:43.658 INFO:teuthology.orchestra.run.smithi047.stdout:Created osd(s) 2 on host 'smithi047' 2024-09-10T18:58:44.226 DEBUG:teuthology.orchestra.run.smithi047:osd.2> sudo journalctl -f -n 0 -u ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968@osd.2.service 2024-09-10T18:58:44.229 INFO:tasks.cephadm:Deploying osd.3 on smithi047 with /dev/vg_nvme/lv_1... 2024-09-10T18:58:44.229 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8563d7e-6fa5-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-09-10T18:58:44.544 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:58:44.963 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:44 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:58:44.963 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:44 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:58:44.963 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:44 smithi047 ceph-mon[39856]: from='osd.2 v2:172.21.15.47:6809/1234745743' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T18:58:44.963 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:44 smithi047 ceph-mon[39856]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T18:58:44.963 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:44 smithi047 ceph-mon[39856]: pgmap v68: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:58:44.963 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:44 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:58:44.963 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:44 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:58:44.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:44 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:58:44.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:44 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:58:44.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:44 smithi047 ceph-mon[29019]: from='osd.2 v2:172.21.15.47:6809/1234745743' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T18:58:44.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:44 smithi047 ceph-mon[29019]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T18:58:44.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:44 smithi047 ceph-mon[29019]: pgmap v68: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:58:44.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:44 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:58:44.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:44 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:58:45.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:44 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:58:45.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:44 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:58:45.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:44 smithi149 ceph-mon[33230]: from='osd.2 v2:172.21.15.47:6809/1234745743' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T18:58:45.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:44 smithi149 ceph-mon[33230]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T18:58:45.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:44 smithi149 ceph-mon[33230]: pgmap v68: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:58:45.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:44 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:58:45.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:44 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:58:45.894 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:45 smithi047 ceph-mon[29019]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-10T18:58:45.894 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:45 smithi047 ceph-mon[29019]: osdmap e16: 3 total, 2 up, 3 in 2024-09-10T18:58:45.894 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:45 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:58:45.894 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:45 smithi047 ceph-mon[29019]: from='osd.2 v2:172.21.15.47:6809/1234745743' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-09-10T18:58:45.895 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:45 smithi047 ceph-mon[29019]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-09-10T18:58:45.895 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:45 smithi047 ceph-mon[39856]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-10T18:58:45.895 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:45 smithi047 ceph-mon[39856]: osdmap e16: 3 total, 2 up, 3 in 2024-09-10T18:58:45.895 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:45 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:58:45.895 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:45 smithi047 ceph-mon[39856]: from='osd.2 v2:172.21.15.47:6809/1234745743' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-09-10T18:58:45.895 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:45 smithi047 ceph-mon[39856]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-09-10T18:58:45.895 INFO:journalctl@ceph.osd.2.smithi047.stdout:Sep 10 18:58:45 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-2[54962]: 2024-09-10T18:58:45.672+0000 7f52150ba640 -1 osd.2 0 waiting for initial osdmap 2024-09-10T18:58:45.896 INFO:journalctl@ceph.osd.2.smithi047.stdout:Sep 10 18:58:45 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-2[54962]: 2024-09-10T18:58:45.675+0000 7f520c6c8640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T18:58:46.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:45 smithi149 ceph-mon[33230]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-10T18:58:46.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:45 smithi149 ceph-mon[33230]: osdmap e16: 3 total, 2 up, 3 in 2024-09-10T18:58:46.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:45 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:58:46.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:45 smithi149 ceph-mon[33230]: from='osd.2 v2:172.21.15.47:6809/1234745743' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-09-10T18:58:46.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:45 smithi149 ceph-mon[33230]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-09-10T18:58:46.338 INFO:teuthology.orchestra.run.smithi047.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-09-10T18:58:46.432 INFO:teuthology.orchestra.run.smithi047.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-09-10T18:58:46.444 INFO:teuthology.orchestra.run.smithi047.stderr: stderr: 10+0 records in 2024-09-10T18:58:46.444 INFO:teuthology.orchestra.run.smithi047.stderr:10+0 records out 2024-09-10T18:58:46.445 INFO:teuthology.orchestra.run.smithi047.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0105542 s, 994 MB/s 2024-09-10T18:58:46.445 INFO:teuthology.orchestra.run.smithi047.stderr:--> Zapping successful for: 2024-09-10T18:58:46.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:46 smithi047 ceph-mon[29019]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-09-10T18:58:46.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:46 smithi047 ceph-mon[29019]: osdmap e17: 3 total, 2 up, 3 in 2024-09-10T18:58:46.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:46 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:58:46.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:46 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:58:46.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:46 smithi047 ceph-mon[29019]: pgmap v71: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:58:46.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:46 smithi047 ceph-mon[39856]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-09-10T18:58:46.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:46 smithi047 ceph-mon[39856]: osdmap e17: 3 total, 2 up, 3 in 2024-09-10T18:58:46.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:46 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:58:46.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:46 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:58:46.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:46 smithi047 ceph-mon[39856]: pgmap v71: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:58:47.092 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:46 smithi149 ceph-mon[33230]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-09-10T18:58:47.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:46 smithi149 ceph-mon[33230]: osdmap e17: 3 total, 2 up, 3 in 2024-09-10T18:58:47.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:46 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:58:47.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:46 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:58:47.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:46 smithi149 ceph-mon[33230]: pgmap v71: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:58:47.818 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8563d7e-6fa5-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi047:vg_nvme/lv_1 2024-09-10T18:58:47.990 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:47 smithi047 ceph-mon[29019]: purged_snaps scrub starts 2024-09-10T18:58:47.990 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:47 smithi047 ceph-mon[29019]: purged_snaps scrub ok 2024-09-10T18:58:47.991 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:47 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:58:47.991 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:47 smithi047 ceph-mon[29019]: osd.2 v2:172.21.15.47:6809/1234745743 boot 2024-09-10T18:58:47.991 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:47 smithi047 ceph-mon[29019]: osdmap e18: 3 total, 3 up, 3 in 2024-09-10T18:58:47.991 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:47 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:58:47.991 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:47 smithi047 ceph-mon[39856]: purged_snaps scrub starts 2024-09-10T18:58:47.991 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:47 smithi047 ceph-mon[39856]: purged_snaps scrub ok 2024-09-10T18:58:47.991 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:47 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:58:47.991 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:47 smithi047 ceph-mon[39856]: osd.2 v2:172.21.15.47:6809/1234745743 boot 2024-09-10T18:58:47.991 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:47 smithi047 ceph-mon[39856]: osdmap e18: 3 total, 3 up, 3 in 2024-09-10T18:58:47.992 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:47 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:58:48.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:47 smithi149 ceph-mon[33230]: purged_snaps scrub starts 2024-09-10T18:58:48.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:47 smithi149 ceph-mon[33230]: purged_snaps scrub ok 2024-09-10T18:58:48.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:47 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:58:48.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:47 smithi149 ceph-mon[33230]: osd.2 v2:172.21.15.47:6809/1234745743 boot 2024-09-10T18:58:48.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:47 smithi149 ceph-mon[33230]: osdmap e18: 3 total, 3 up, 3 in 2024-09-10T18:58:48.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:47 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:58:48.618 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:58:48.681 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:48 smithi047 ceph-mon[29019]: osdmap e19: 3 total, 3 up, 3 in 2024-09-10T18:58:48.932 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:48 smithi047 ceph-mon[39856]: osdmap e19: 3 total, 3 up, 3 in 2024-09-10T18:58:48.932 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:48 smithi047 ceph-mon[39856]: pgmap v74: 0 pgs: ; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:58:48.932 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:48 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-09-10T18:58:48.933 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:48 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:58:48.933 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:48 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:58:48.933 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:48 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:58:48.933 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:48 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:58:48.933 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:48 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:58:48.933 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:48 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:58:48.933 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:48 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:58:48.933 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:48 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:58:48.933 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:48 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:58:48.933 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:48 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:58:48.933 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:48 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:58:48.933 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:48 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:58:48.934 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:48 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:58:48.934 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:48 smithi047 ceph-mon[29019]: pgmap v74: 0 pgs: ; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:58:48.934 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:48 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-09-10T18:58:48.934 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:48 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:58:48.934 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:48 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:58:48.934 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:48 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:58:48.934 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:48 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:58:48.934 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:48 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:58:48.934 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:48 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:58:48.934 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:48 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:58:48.935 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:48 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:58:48.935 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:48 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:58:48.935 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:48 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:58:48.935 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:48 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:58:48.935 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:48 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:58:48.935 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:48 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:58:49.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:48 smithi149 ceph-mon[33230]: osdmap e19: 3 total, 3 up, 3 in 2024-09-10T18:58:49.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:48 smithi149 ceph-mon[33230]: pgmap v74: 0 pgs: ; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:58:49.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:48 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-09-10T18:58:49.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:48 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:58:49.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:48 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:58:49.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:48 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:58:49.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:48 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:58:49.094 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:48 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:58:49.094 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:48 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:58:49.094 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:48 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:58:49.094 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:48 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:58:49.094 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:48 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:58:49.094 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:48 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:58:49.094 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:48 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:58:49.094 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:48 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:58:49.094 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:48 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:58:49.994 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:49 smithi149 ceph-mon[33230]: Detected new or changed devices on smithi047 2024-09-10T18:58:49.994 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:49 smithi149 ceph-mon[33230]: Adjusting osd_memory_target on smithi047 to 5356M 2024-09-10T18:58:49.994 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:49 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-09-10T18:58:49.994 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:49 smithi149 ceph-mon[33230]: osdmap e20: 3 total, 3 up, 3 in 2024-09-10T18:58:49.994 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:49 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:58:50.004 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:49 smithi047 ceph-mon[29019]: Detected new or changed devices on smithi047 2024-09-10T18:58:50.005 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:49 smithi047 ceph-mon[29019]: Adjusting osd_memory_target on smithi047 to 5356M 2024-09-10T18:58:50.005 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:49 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-09-10T18:58:50.005 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:49 smithi047 ceph-mon[29019]: osdmap e20: 3 total, 3 up, 3 in 2024-09-10T18:58:50.005 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:49 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:58:50.005 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:49 smithi047 ceph-mon[39856]: Detected new or changed devices on smithi047 2024-09-10T18:58:50.005 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:49 smithi047 ceph-mon[39856]: Adjusting osd_memory_target on smithi047 to 5356M 2024-09-10T18:58:50.005 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:49 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-09-10T18:58:50.005 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:49 smithi047 ceph-mon[39856]: osdmap e20: 3 total, 3 up, 3 in 2024-09-10T18:58:50.005 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:49 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:58:50.520 INFO:journalctl@ceph.osd.0.smithi047.stdout:Sep 10 18:58:50 smithi047 sudo[58827]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T18:58:50.520 INFO:journalctl@ceph.osd.0.smithi047.stdout:Sep 10 18:58:50 smithi047 sudo[58827]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T18:58:50.520 INFO:journalctl@ceph.osd.0.smithi047.stdout:Sep 10 18:58:50 smithi047 sudo[58827]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T18:58:50.520 INFO:journalctl@ceph.osd.0.smithi047.stdout:Sep 10 18:58:50 smithi047 sudo[58827]: pam_unix(sudo:session): session closed for user root 2024-09-10T18:58:50.789 INFO:journalctl@ceph.osd.0.smithi047.stdout:Sep 10 18:58:50 smithi047 sudo[58925]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T18:58:50.789 INFO:journalctl@ceph.osd.0.smithi047.stdout:Sep 10 18:58:50 smithi047 sudo[58925]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T18:58:50.790 INFO:journalctl@ceph.osd.0.smithi047.stdout:Sep 10 18:58:50 smithi047 sudo[58925]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T18:58:50.790 INFO:journalctl@ceph.osd.0.smithi047.stdout:Sep 10 18:58:50 smithi047 sudo[58925]: pam_unix(sudo:session): session closed for user root 2024-09-10T18:58:50.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:50 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-10T18:58:50.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:50 smithi047 ceph-mon[29019]: osdmap e21: 3 total, 3 up, 3 in 2024-09-10T18:58:50.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:50 smithi047 ceph-mon[29019]: pgmap v77: 1 pgs: 1 creating+peering; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:58:50.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:50 smithi047 ceph-mon[29019]: from='client.14304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:58:50.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:50 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:58:50.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:50 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:58:50.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:50 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:58:50.791 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:50 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-10T18:58:50.791 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:50 smithi047 ceph-mon[39856]: osdmap e21: 3 total, 3 up, 3 in 2024-09-10T18:58:50.791 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:50 smithi047 ceph-mon[39856]: pgmap v77: 1 pgs: 1 creating+peering; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:58:50.791 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:50 smithi047 ceph-mon[39856]: from='client.14304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:58:50.791 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:50 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:58:50.791 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:50 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:58:50.791 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:50 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:58:51.092 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:50 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-10T18:58:51.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:50 smithi149 ceph-mon[33230]: osdmap e21: 3 total, 3 up, 3 in 2024-09-10T18:58:51.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:50 smithi149 ceph-mon[33230]: pgmap v77: 1 pgs: 1 creating+peering; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:58:51.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:50 smithi149 ceph-mon[33230]: from='client.14304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:58:51.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:50 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:58:51.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:50 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:58:51.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:50 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:58:51.539 INFO:journalctl@ceph.osd.1.smithi047.stdout:Sep 10 18:58:51 smithi047 sudo[58954]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T18:58:51.539 INFO:journalctl@ceph.osd.1.smithi047.stdout:Sep 10 18:58:51 smithi047 sudo[58954]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T18:58:51.540 INFO:journalctl@ceph.osd.1.smithi047.stdout:Sep 10 18:58:51 smithi047 sudo[58954]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T18:58:51.540 INFO:journalctl@ceph.osd.1.smithi047.stdout:Sep 10 18:58:51 smithi047 sudo[58954]: pam_unix(sudo:session): session closed for user root 2024-09-10T18:58:51.871 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:51 smithi047 ceph-mon[39856]: osdmap e22: 3 total, 3 up, 3 in 2024-09-10T18:58:51.871 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:51 smithi047 ceph-mon[39856]: mgrmap e15: y(active, since 2m), standbys: x 2024-09-10T18:58:51.871 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:51 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:58:51.871 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:51 smithi047 ceph-mon[29019]: osdmap e22: 3 total, 3 up, 3 in 2024-09-10T18:58:51.871 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:51 smithi047 ceph-mon[29019]: mgrmap e15: y(active, since 2m), standbys: x 2024-09-10T18:58:51.871 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:51 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:58:51.871 INFO:journalctl@ceph.osd.1.smithi047.stdout:Sep 10 18:58:51 smithi047 sudo[58981]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T18:58:51.872 INFO:journalctl@ceph.osd.1.smithi047.stdout:Sep 10 18:58:51 smithi047 sudo[58981]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T18:58:51.872 INFO:journalctl@ceph.osd.1.smithi047.stdout:Sep 10 18:58:51 smithi047 sudo[58981]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T18:58:51.872 INFO:journalctl@ceph.osd.1.smithi047.stdout:Sep 10 18:58:51 smithi047 sudo[58981]: pam_unix(sudo:session): session closed for user root 2024-09-10T18:58:52.092 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:51 smithi149 ceph-mon[33230]: osdmap e22: 3 total, 3 up, 3 in 2024-09-10T18:58:52.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:51 smithi149 ceph-mon[33230]: mgrmap e15: y(active, since 2m), standbys: x 2024-09-10T18:58:52.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:51 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:58:52.130 INFO:journalctl@ceph.osd.2.smithi047.stdout:Sep 10 18:58:52 smithi047 sudo[58992]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T18:58:52.131 INFO:journalctl@ceph.osd.2.smithi047.stdout:Sep 10 18:58:52 smithi047 sudo[58992]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T18:58:52.131 INFO:journalctl@ceph.osd.2.smithi047.stdout:Sep 10 18:58:52 smithi047 sudo[58992]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T18:58:52.131 INFO:journalctl@ceph.osd.2.smithi047.stdout:Sep 10 18:58:52 smithi047 sudo[58992]: pam_unix(sudo:session): session closed for user root 2024-09-10T18:58:52.927 INFO:journalctl@ceph.osd.2.smithi047.stdout:Sep 10 18:58:52 smithi047 sudo[59037]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T18:58:52.927 INFO:journalctl@ceph.osd.2.smithi047.stdout:Sep 10 18:58:52 smithi047 sudo[59037]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T18:58:52.927 INFO:journalctl@ceph.osd.2.smithi047.stdout:Sep 10 18:58:52 smithi047 sudo[59037]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T18:58:52.927 INFO:journalctl@ceph.osd.2.smithi047.stdout:Sep 10 18:58:52 smithi047 sudo[59037]: pam_unix(sudo:session): session closed for user root 2024-09-10T18:58:53.263 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:53 smithi047 sudo[59053]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T18:58:53.263 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:53 smithi047 sudo[59053]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T18:58:53.263 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:53 smithi047 sudo[59053]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T18:58:53.264 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:53 smithi047 sudo[59053]: pam_unix(sudo:session): session closed for user root 2024-09-10T18:58:53.539 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:53 smithi047 ceph-mon[39856]: pgmap v79: 1 pgs: 1 creating+peering; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:58:53.539 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:53 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/1250067162' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "126cbc1e-5b4f-439f-9d03-49018692c2a8"}]: dispatch 2024-09-10T18:58:53.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:53 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/1250067162' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "126cbc1e-5b4f-439f-9d03-49018692c2a8"}]': finished 2024-09-10T18:58:53.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:53 smithi047 ceph-mon[39856]: osdmap e23: 4 total, 3 up, 4 in 2024-09-10T18:58:53.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:53 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:58:53.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:53 smithi047 ceph-mon[39856]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T18:58:53.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:53 smithi047 sudo[59121]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T18:58:53.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:53 smithi047 sudo[59121]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T18:58:53.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:53 smithi047 sudo[59121]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T18:58:53.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:53 smithi047 sudo[59121]: pam_unix(sudo:session): session closed for user root 2024-09-10T18:58:53.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:53 smithi047 ceph-mon[29019]: pgmap v79: 1 pgs: 1 creating+peering; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:58:53.541 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:53 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/1250067162' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "126cbc1e-5b4f-439f-9d03-49018692c2a8"}]: dispatch 2024-09-10T18:58:53.541 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:53 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/1250067162' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "126cbc1e-5b4f-439f-9d03-49018692c2a8"}]': finished 2024-09-10T18:58:53.541 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:53 smithi047 ceph-mon[29019]: osdmap e23: 4 total, 3 up, 4 in 2024-09-10T18:58:53.541 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:53 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:58:53.541 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:53 smithi047 ceph-mon[29019]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T18:58:53.745 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:53 smithi149 ceph-mon[33230]: pgmap v79: 1 pgs: 1 creating+peering; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:58:53.745 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:53 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/1250067162' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "126cbc1e-5b4f-439f-9d03-49018692c2a8"}]: dispatch 2024-09-10T18:58:53.746 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:53 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/1250067162' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "126cbc1e-5b4f-439f-9d03-49018692c2a8"}]': finished 2024-09-10T18:58:53.746 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:53 smithi149 ceph-mon[33230]: osdmap e23: 4 total, 3 up, 4 in 2024-09-10T18:58:53.746 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:53 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:58:53.746 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:53 smithi149 ceph-mon[33230]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T18:58:53.746 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:53 smithi149 sudo[35737]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T18:58:53.746 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:53 smithi149 sudo[35737]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T18:58:53.746 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:53 smithi149 sudo[35737]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T18:58:53.746 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:53 smithi149 sudo[35737]: pam_unix(sudo:session): session closed for user root 2024-09-10T18:58:54.092 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:53 smithi149 sudo[35740]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T18:58:54.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:53 smithi149 sudo[35740]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T18:58:54.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:53 smithi149 sudo[35740]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T18:58:54.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:53 smithi149 sudo[35740]: pam_unix(sudo:session): session closed for user root 2024-09-10T18:58:54.789 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:54 smithi047 sudo[59499]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T18:58:54.789 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:54 smithi047 sudo[59499]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T18:58:54.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:54 smithi047 sudo[59499]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T18:58:54.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:54 smithi047 sudo[59499]: pam_unix(sudo:session): session closed for user root 2024-09-10T18:58:55.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:54 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/3949596078' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:58:55.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:54 smithi047 ceph-mon[29019]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T18:58:55.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:54 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:58:55.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:54 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:58:55.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:54 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:58:55.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:54 smithi047 ceph-mon[29019]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T18:58:55.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:54 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:58:55.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:54 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:58:55.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:54 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:58:55.291 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:54 smithi047 ceph-mon[29019]: pgmap v81: 1 pgs: 1 creating+peering; 0 B data, 873 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:58:55.291 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:54 smithi047 ceph-mon[29019]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T18:58:55.291 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:54 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:58:55.291 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:54 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:58:55.291 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:54 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:58:55.291 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:54 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:58:55.291 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:54 smithi047 sudo[59881]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T18:58:55.291 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:54 smithi047 sudo[59881]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T18:58:55.291 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:54 smithi047 sudo[59881]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T18:58:55.291 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:54 smithi047 sudo[59881]: pam_unix(sudo:session): session closed for user root 2024-09-10T18:58:55.292 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:54 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/3949596078' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:58:55.292 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:54 smithi047 ceph-mon[39856]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T18:58:55.292 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:54 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:58:55.292 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:54 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:58:55.292 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:54 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:58:55.292 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:54 smithi047 ceph-mon[39856]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T18:58:55.292 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:54 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:58:55.292 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:54 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:58:55.292 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:54 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:58:55.292 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:54 smithi047 ceph-mon[39856]: pgmap v81: 1 pgs: 1 creating+peering; 0 B data, 873 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:58:55.292 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:54 smithi047 ceph-mon[39856]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T18:58:55.292 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:54 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:58:55.292 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:54 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:58:55.293 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:54 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:58:55.293 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:54 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:58:55.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:54 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/3949596078' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:58:55.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:54 smithi149 ceph-mon[33230]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T18:58:55.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:54 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:58:55.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:54 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:58:55.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:54 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:58:55.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:54 smithi149 ceph-mon[33230]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T18:58:55.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:54 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:58:55.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:54 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:58:55.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:54 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:58:55.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:54 smithi149 ceph-mon[33230]: pgmap v81: 1 pgs: 1 creating+peering; 0 B data, 873 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:58:55.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:54 smithi149 ceph-mon[33230]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T18:58:55.344 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:54 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:58:55.344 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:54 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:58:55.344 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:54 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:58:55.344 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:54 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:58:56.039 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:55 smithi047 ceph-mon[29019]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T18:58:56.039 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:55 smithi047 ceph-mon[29019]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T18:58:56.040 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:55 smithi047 ceph-mon[39856]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T18:58:56.040 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:55 smithi047 ceph-mon[39856]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T18:58:56.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:55 smithi149 ceph-mon[33230]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T18:58:56.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:55 smithi149 ceph-mon[33230]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T18:58:57.039 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:56 smithi047 ceph-mon[29019]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 873 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:58:57.039 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:56 smithi047 ceph-mon[39856]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 873 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:58:57.342 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:56 smithi149 ceph-mon[33230]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 873 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:58:59.265 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:59 smithi047 ceph-mon[29019]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:58:59.265 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:59 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-10T18:58:59.265 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:58:59 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:58:59.265 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:59 smithi047 ceph-mon[39856]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:58:59.265 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:59 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-10T18:58:59.266 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:58:59 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:58:59.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:59 smithi149 ceph-mon[33230]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:58:59.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:59 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-10T18:58:59.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:58:59 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:59:00.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:00 smithi047 ceph-mon[29019]: Deploying daemon osd.3 on smithi047 2024-09-10T18:59:00.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:00 smithi047 ceph-mon[39856]: Deploying daemon osd.3 on smithi047 2024-09-10T18:59:00.342 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:00 smithi149 ceph-mon[33230]: Deploying daemon osd.3 on smithi047 2024-09-10T18:59:01.096 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:01 smithi047 ceph-mon[29019]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.2 KiB/s rd, 65 KiB/s wr, 5 op/s 2024-09-10T18:59:01.096 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:01 smithi047 ceph-mon[39856]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.2 KiB/s rd, 65 KiB/s wr, 5 op/s 2024-09-10T18:59:01.342 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:01 smithi149 ceph-mon[33230]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.2 KiB/s rd, 65 KiB/s wr, 5 op/s 2024-09-10T18:59:03.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:03 smithi047 ceph-mon[29019]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-09-10T18:59:03.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:03 smithi047 ceph-mon[39856]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-09-10T18:59:03.342 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:03 smithi149 ceph-mon[33230]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-09-10T18:59:04.211 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:04 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:59:04.211 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:04 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:04.211 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:04 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:04.211 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:04 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:59:04.211 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:04 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:04.211 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:04 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:04.342 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:04 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:59:04.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:04 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:04.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:04 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:05.242 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:05 smithi047 ceph-mon[29019]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-09-10T18:59:05.242 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:05 smithi047 ceph-mon[39856]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-09-10T18:59:05.342 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:05 smithi149 ceph-mon[33230]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-09-10T18:59:06.837 INFO:teuthology.orchestra.run.smithi047.stdout:Created osd(s) 3 on host 'smithi047' 2024-09-10T18:59:07.368 DEBUG:teuthology.orchestra.run.smithi047:osd.3> sudo journalctl -f -n 0 -u ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968@osd.3.service 2024-09-10T18:59:07.371 INFO:tasks.cephadm:Deploying osd.4 on smithi149 with /dev/vg_nvme/lv_4... 2024-09-10T18:59:07.371 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8563d7e-6fa5-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-09-10T18:59:07.392 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:07 smithi047 ceph-mon[29019]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-10T18:59:07.392 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:07 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:07.392 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:07 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:07.392 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:07 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:59:07.393 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:07 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:59:07.393 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:07 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:07.393 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:07 smithi047 ceph-mon[29019]: from='osd.3 v2:172.21.15.47:6813/2685917187' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-10T18:59:07.393 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:07 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:59:07.393 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:07 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:07.393 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:07 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:07.393 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:07 smithi047 ceph-mon[39856]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-10T18:59:07.393 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:07 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:07.393 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:07 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:07.393 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:07 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:59:07.393 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:07 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:59:07.393 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:07 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:07.394 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:07 smithi047 ceph-mon[39856]: from='osd.3 v2:172.21.15.47:6813/2685917187' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-10T18:59:07.394 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:07 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:59:07.394 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:07 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:07.394 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:07 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:07.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:07 smithi149 ceph-mon[33230]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-10T18:59:07.596 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:07 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:07.596 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:07 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:07.596 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:07 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:59:07.596 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:07 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:59:07.596 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:07 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:07.596 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:07 smithi149 ceph-mon[33230]: from='osd.3 v2:172.21.15.47:6813/2685917187' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-10T18:59:07.596 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:07 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:59:07.596 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:07 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:07.596 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:07 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:07.620 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:59:08.342 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:08 smithi149 ceph-mon[33230]: from='osd.3 v2:172.21.15.47:6813/2685917187' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-10T18:59:08.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:08 smithi149 ceph-mon[33230]: osdmap e24: 4 total, 3 up, 4 in 2024-09-10T18:59:08.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:08 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:59:08.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:08 smithi149 ceph-mon[33230]: from='osd.3 v2:172.21.15.47:6813/2685917187' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-09-10T18:59:08.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:08 smithi047 ceph-mon[29019]: from='osd.3 v2:172.21.15.47:6813/2685917187' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-10T18:59:08.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:08 smithi047 ceph-mon[29019]: osdmap e24: 4 total, 3 up, 4 in 2024-09-10T18:59:08.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:08 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:59:08.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:08 smithi047 ceph-mon[29019]: from='osd.3 v2:172.21.15.47:6813/2685917187' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-09-10T18:59:08.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:08 smithi047 ceph-mon[39856]: from='osd.3 v2:172.21.15.47:6813/2685917187' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-10T18:59:08.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:08 smithi047 ceph-mon[39856]: osdmap e24: 4 total, 3 up, 4 in 2024-09-10T18:59:08.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:08 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:59:08.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:08 smithi047 ceph-mon[39856]: from='osd.3 v2:172.21.15.47:6813/2685917187' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-09-10T18:59:08.541 INFO:journalctl@ceph.osd.3.smithi047.stdout:Sep 10 18:59:08 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-3[60718]: 2024-09-10T18:59:08.174+0000 7f0665c24640 -1 osd.3 0 waiting for initial osdmap 2024-09-10T18:59:08.541 INFO:journalctl@ceph.osd.3.smithi047.stdout:Sep 10 18:59:08 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-3[60718]: 2024-09-10T18:59:08.180+0000 7f066125c640 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T18:59:09.264 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-09-10T18:59:09.351 INFO:teuthology.orchestra.run.smithi149.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-09-10T18:59:09.364 INFO:teuthology.orchestra.run.smithi149.stderr: stderr: 10+0 records in 2024-09-10T18:59:09.364 INFO:teuthology.orchestra.run.smithi149.stderr:10+0 records out 2024-09-10T18:59:09.364 INFO:teuthology.orchestra.run.smithi149.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113473 s, 924 MB/s 2024-09-10T18:59:09.365 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping successful for: 2024-09-10T18:59:09.453 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:09 smithi149 ceph-mon[33230]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-10T18:59:09.453 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:09 smithi149 ceph-mon[33230]: from='osd.3 v2:172.21.15.47:6813/2685917187' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-09-10T18:59:09.453 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:09 smithi149 ceph-mon[33230]: osdmap e25: 4 total, 3 up, 4 in 2024-09-10T18:59:09.453 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:09 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:59:09.453 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:09 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:59:09.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:09 smithi047 ceph-mon[29019]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-10T18:59:09.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:09 smithi047 ceph-mon[29019]: from='osd.3 v2:172.21.15.47:6813/2685917187' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-09-10T18:59:09.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:09 smithi047 ceph-mon[29019]: osdmap e25: 4 total, 3 up, 4 in 2024-09-10T18:59:09.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:09 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:59:09.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:09 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:59:09.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:09 smithi047 ceph-mon[39856]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-10T18:59:09.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:09 smithi047 ceph-mon[39856]: from='osd.3 v2:172.21.15.47:6813/2685917187' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-09-10T18:59:09.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:09 smithi047 ceph-mon[39856]: osdmap e25: 4 total, 3 up, 4 in 2024-09-10T18:59:09.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:09 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:59:09.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:09 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:59:09.981 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8563d7e-6fa5-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi149:vg_nvme/lv_4 2024-09-10T18:59:10.227 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:59:10.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:10 smithi149 ceph-mon[33230]: purged_snaps scrub starts 2024-09-10T18:59:10.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:10 smithi149 ceph-mon[33230]: purged_snaps scrub ok 2024-09-10T18:59:10.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:10 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:59:10.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:10 smithi149 ceph-mon[33230]: osd.3 v2:172.21.15.47:6813/2685917187 boot 2024-09-10T18:59:10.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:10 smithi149 ceph-mon[33230]: osdmap e26: 4 total, 4 up, 4 in 2024-09-10T18:59:10.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:10 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:59:10.539 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:10 smithi047 ceph-mon[39856]: purged_snaps scrub starts 2024-09-10T18:59:10.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:10 smithi047 ceph-mon[39856]: purged_snaps scrub ok 2024-09-10T18:59:10.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:10 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:59:10.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:10 smithi047 ceph-mon[39856]: osd.3 v2:172.21.15.47:6813/2685917187 boot 2024-09-10T18:59:10.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:10 smithi047 ceph-mon[39856]: osdmap e26: 4 total, 4 up, 4 in 2024-09-10T18:59:10.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:10 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:59:10.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:10 smithi047 ceph-mon[29019]: purged_snaps scrub starts 2024-09-10T18:59:10.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:10 smithi047 ceph-mon[29019]: purged_snaps scrub ok 2024-09-10T18:59:10.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:10 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:59:10.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:10 smithi047 ceph-mon[29019]: osd.3 v2:172.21.15.47:6813/2685917187 boot 2024-09-10T18:59:10.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:10 smithi047 ceph-mon[29019]: osdmap e26: 4 total, 4 up, 4 in 2024-09-10T18:59:10.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:10 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:59:11.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:11 smithi149 ceph-mon[33230]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:59:11.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:11 smithi149 ceph-mon[33230]: osdmap e27: 4 total, 4 up, 4 in 2024-09-10T18:59:11.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:11 smithi149 ceph-mon[33230]: Detected new or changed devices on smithi047 2024-09-10T18:59:11.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:11 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:11.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:11 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:11.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:11 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:59:11.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:11 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:59:11.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:11 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:59:11.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:11 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:59:11.344 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:11 smithi149 ceph-mon[33230]: Adjusting osd_memory_target on smithi047 to 4017M 2024-09-10T18:59:11.344 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:11 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:11.344 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:11 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:59:11.344 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:11 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:59:11.344 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:11 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:11.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:11 smithi047 ceph-mon[29019]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:59:11.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:11 smithi047 ceph-mon[29019]: osdmap e27: 4 total, 4 up, 4 in 2024-09-10T18:59:11.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:11 smithi047 ceph-mon[29019]: Detected new or changed devices on smithi047 2024-09-10T18:59:11.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:11 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:11.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:11 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:11.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:11 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:59:11.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:11 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:59:11.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:11 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:59:11.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:11 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:59:11.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:11 smithi047 ceph-mon[29019]: Adjusting osd_memory_target on smithi047 to 4017M 2024-09-10T18:59:11.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:11 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:11.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:11 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:59:11.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:11 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:59:11.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:11 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:11.541 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:11 smithi047 ceph-mon[39856]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:59:11.541 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:11 smithi047 ceph-mon[39856]: osdmap e27: 4 total, 4 up, 4 in 2024-09-10T18:59:11.541 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:11 smithi047 ceph-mon[39856]: Detected new or changed devices on smithi047 2024-09-10T18:59:11.541 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:11 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:11.541 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:11 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:11.541 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:11 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:59:11.541 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:11 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:59:11.541 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:11 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:59:11.541 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:11 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:59:11.541 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:11 smithi047 ceph-mon[39856]: Adjusting osd_memory_target on smithi047 to 4017M 2024-09-10T18:59:11.541 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:11 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:11.541 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:11 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:59:11.542 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:11 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:59:11.542 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:11 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:12.478 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:12 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:59:12.478 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:12 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:59:12.478 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:12 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:59:12.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:12 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:59:12.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:12 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:59:12.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:12 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:59:12.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:12 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:59:12.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:12 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:59:12.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:12 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:59:13.206 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:13 smithi149 ceph-mon[33230]: from='client.24187 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:59:13.206 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:13 smithi149 ceph-mon[33230]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:59:13.541 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:13 smithi047 ceph-mon[29019]: from='client.24187 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:59:13.541 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:13 smithi047 ceph-mon[29019]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:59:13.541 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:13 smithi047 ceph-mon[39856]: from='client.24187 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:59:13.541 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:13 smithi047 ceph-mon[39856]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:59:14.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:14 smithi047 ceph-mon[29019]: from='client.? 172.21.15.149:0/3122752196' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4a83f8dc-b265-4f60-a8e1-8bcc6fd4130c"}]: dispatch 2024-09-10T18:59:14.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:14 smithi047 ceph-mon[29019]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4a83f8dc-b265-4f60-a8e1-8bcc6fd4130c"}]: dispatch 2024-09-10T18:59:14.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:14 smithi047 ceph-mon[29019]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4a83f8dc-b265-4f60-a8e1-8bcc6fd4130c"}]': finished 2024-09-10T18:59:14.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:14 smithi047 ceph-mon[29019]: osdmap e28: 5 total, 4 up, 5 in 2024-09-10T18:59:14.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:14 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:59:14.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:14 smithi047 ceph-mon[39856]: from='client.? 172.21.15.149:0/3122752196' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4a83f8dc-b265-4f60-a8e1-8bcc6fd4130c"}]: dispatch 2024-09-10T18:59:14.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:14 smithi047 ceph-mon[39856]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4a83f8dc-b265-4f60-a8e1-8bcc6fd4130c"}]: dispatch 2024-09-10T18:59:14.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:14 smithi047 ceph-mon[39856]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4a83f8dc-b265-4f60-a8e1-8bcc6fd4130c"}]': finished 2024-09-10T18:59:14.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:14 smithi047 ceph-mon[39856]: osdmap e28: 5 total, 4 up, 5 in 2024-09-10T18:59:14.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:14 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:59:14.592 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:14 smithi149 ceph-mon[33230]: from='client.? 172.21.15.149:0/3122752196' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4a83f8dc-b265-4f60-a8e1-8bcc6fd4130c"}]: dispatch 2024-09-10T18:59:14.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:14 smithi149 ceph-mon[33230]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4a83f8dc-b265-4f60-a8e1-8bcc6fd4130c"}]: dispatch 2024-09-10T18:59:14.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:14 smithi149 ceph-mon[33230]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4a83f8dc-b265-4f60-a8e1-8bcc6fd4130c"}]': finished 2024-09-10T18:59:14.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:14 smithi149 ceph-mon[33230]: osdmap e28: 5 total, 4 up, 5 in 2024-09-10T18:59:14.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:14 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:59:15.539 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:15 smithi047 ceph-mon[39856]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:59:15.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:15 smithi047 ceph-mon[39856]: from='client.? 172.21.15.149:0/1190928130' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:59:15.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:15 smithi047 ceph-mon[29019]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:59:15.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:15 smithi047 ceph-mon[29019]: from='client.? 172.21.15.149:0/1190928130' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:59:15.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:15 smithi149 ceph-mon[33230]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:59:15.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:15 smithi149 ceph-mon[33230]: from='client.? 172.21.15.149:0/1190928130' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:59:17.473 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:17 smithi149 ceph-mon[33230]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:59:17.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:17 smithi047 ceph-mon[29019]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:59:17.539 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:17 smithi047 ceph-mon[39856]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:59:19.215 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:19 smithi149 ceph-mon[33230]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:59:19.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:19 smithi047 ceph-mon[29019]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:59:19.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:19 smithi047 ceph-mon[39856]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:59:20.511 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:20 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-10T18:59:20.511 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:20 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:59:20.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:20 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-10T18:59:20.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:20 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:59:20.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:20 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-10T18:59:20.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:20 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:59:21.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:21 smithi047 ceph-mon[29019]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:59:21.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:21 smithi047 ceph-mon[29019]: Deploying daemon osd.4 on smithi149 2024-09-10T18:59:21.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:21 smithi047 ceph-mon[39856]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:59:21.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:21 smithi047 ceph-mon[39856]: Deploying daemon osd.4 on smithi149 2024-09-10T18:59:21.582 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:21 smithi149 ceph-mon[33230]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:59:21.582 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:21 smithi149 ceph-mon[33230]: Deploying daemon osd.4 on smithi149 2024-09-10T18:59:23.502 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:23 smithi149 ceph-mon[33230]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:59:23.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:23 smithi047 ceph-mon[29019]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:59:23.539 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:23 smithi047 ceph-mon[39856]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:59:25.494 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:25 smithi149 ceph-mon[33230]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:59:25.495 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:25 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:59:25.495 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:25 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:25.495 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:25 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:25.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:25 smithi047 ceph-mon[29019]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:59:25.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:25 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:59:25.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:25 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:25.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:25 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:25.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:25 smithi047 ceph-mon[39856]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:59:25.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:25 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:59:25.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:25 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:25.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:25 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:27.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:27 smithi047 ceph-mon[29019]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:59:27.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:27 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:27.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:27 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:27.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:27 smithi047 ceph-mon[39856]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:59:27.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:27 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:27.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:27 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:27.592 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:27 smithi149 ceph-mon[33230]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:59:27.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:27 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:27.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:27 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:27.985 INFO:teuthology.orchestra.run.smithi149.stdout:Created osd(s) 4 on host 'smithi149' 2024-09-10T18:59:28.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:28 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:59:28.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:28 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:59:28.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:28 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:28.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:28 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:59:28.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:28 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:28.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:28 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:28.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:28 smithi047 ceph-mon[29019]: from='osd.4 v2:172.21.15.149:6800/2978707630' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T18:59:28.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:28 smithi047 ceph-mon[29019]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T18:59:28.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:28 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:59:28.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:28 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:59:28.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:28 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:28.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:28 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:59:28.541 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:28 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:28.541 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:28 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:28.541 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:28 smithi047 ceph-mon[39856]: from='osd.4 v2:172.21.15.149:6800/2978707630' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T18:59:28.541 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:28 smithi047 ceph-mon[39856]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T18:59:28.592 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:28 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:59:28.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:28 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:59:28.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:28 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:28.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:28 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:59:28.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:28 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:28.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:28 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:28.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:28 smithi149 ceph-mon[33230]: from='osd.4 v2:172.21.15.149:6800/2978707630' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T18:59:28.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:28 smithi149 ceph-mon[33230]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T18:59:28.619 DEBUG:teuthology.orchestra.run.smithi149:osd.4> sudo journalctl -f -n 0 -u ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968@osd.4.service 2024-09-10T18:59:28.621 INFO:tasks.cephadm:Deploying osd.5 on smithi149 with /dev/vg_nvme/lv_3... 2024-09-10T18:59:28.622 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8563d7e-6fa5-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-09-10T18:59:29.539 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:29 smithi047 ceph-mon[39856]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:59:29.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:29 smithi047 ceph-mon[39856]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-10T18:59:29.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:29 smithi047 ceph-mon[39856]: osdmap e29: 5 total, 4 up, 5 in 2024-09-10T18:59:29.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:29 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:59:29.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:29 smithi047 ceph-mon[39856]: from='osd.4 v2:172.21.15.149:6800/2978707630' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-09-10T18:59:29.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:29 smithi047 ceph-mon[39856]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-09-10T18:59:29.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:29 smithi047 ceph-mon[29019]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:59:29.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:29 smithi047 ceph-mon[29019]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-10T18:59:29.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:29 smithi047 ceph-mon[29019]: osdmap e29: 5 total, 4 up, 5 in 2024-09-10T18:59:29.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:29 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:59:29.541 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:29 smithi047 ceph-mon[29019]: from='osd.4 v2:172.21.15.149:6800/2978707630' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-09-10T18:59:29.541 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:29 smithi047 ceph-mon[29019]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-09-10T18:59:29.575 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:29 smithi149 ceph-mon[33230]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:59:29.575 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:29 smithi149 ceph-mon[33230]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-10T18:59:29.575 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:29 smithi149 ceph-mon[33230]: osdmap e29: 5 total, 4 up, 5 in 2024-09-10T18:59:29.575 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:29 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:59:29.575 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:29 smithi149 ceph-mon[33230]: from='osd.4 v2:172.21.15.149:6800/2978707630' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-09-10T18:59:29.576 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:29 smithi149 ceph-mon[33230]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-09-10T18:59:29.622 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:59:30.093 INFO:journalctl@ceph.osd.4.smithi149.stdout:Sep 10 18:59:29 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-4[38043]: 2024-09-10T18:59:29.988+0000 7f4eaa6dc640 -1 osd.4 0 waiting for initial osdmap 2024-09-10T18:59:30.093 INFO:journalctl@ceph.osd.4.smithi149.stdout:Sep 10 18:59:29 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-4[38043]: 2024-09-10T18:59:29.992+0000 7f4ea350f640 -1 osd.4 30 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T18:59:31.277 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:30 smithi149 ceph-mon[33230]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:59:31.278 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:30 smithi149 ceph-mon[33230]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-09-10T18:59:31.278 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:30 smithi149 ceph-mon[33230]: osdmap e30: 5 total, 4 up, 5 in 2024-09-10T18:59:31.278 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:30 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:59:31.278 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:30 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:59:31.289 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:30 smithi047 ceph-mon[39856]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:59:31.289 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:30 smithi047 ceph-mon[39856]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-09-10T18:59:31.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:30 smithi047 ceph-mon[39856]: osdmap e30: 5 total, 4 up, 5 in 2024-09-10T18:59:31.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:30 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:59:31.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:30 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:59:31.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:30 smithi047 ceph-mon[29019]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:59:31.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:30 smithi047 ceph-mon[29019]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-09-10T18:59:31.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:30 smithi047 ceph-mon[29019]: osdmap e30: 5 total, 4 up, 5 in 2024-09-10T18:59:31.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:30 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:59:31.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:30 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:59:31.853 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-09-10T18:59:31.941 INFO:teuthology.orchestra.run.smithi149.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-09-10T18:59:31.954 INFO:teuthology.orchestra.run.smithi149.stderr: stderr: 10+0 records in 2024-09-10T18:59:31.954 INFO:teuthology.orchestra.run.smithi149.stderr:10+0 records out 2024-09-10T18:59:31.954 INFO:teuthology.orchestra.run.smithi149.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0115134 s, 911 MB/s 2024-09-10T18:59:31.954 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping successful for: 2024-09-10T18:59:32.289 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:31 smithi047 ceph-mon[29019]: purged_snaps scrub starts 2024-09-10T18:59:32.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:31 smithi047 ceph-mon[29019]: purged_snaps scrub ok 2024-09-10T18:59:32.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:31 smithi047 ceph-mon[29019]: osd.4 v2:172.21.15.149:6800/2978707630 boot 2024-09-10T18:59:32.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:31 smithi047 ceph-mon[29019]: osdmap e31: 5 total, 5 up, 5 in 2024-09-10T18:59:32.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:31 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:59:32.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:31 smithi047 ceph-mon[39856]: purged_snaps scrub starts 2024-09-10T18:59:32.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:31 smithi047 ceph-mon[39856]: purged_snaps scrub ok 2024-09-10T18:59:32.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:31 smithi047 ceph-mon[39856]: osd.4 v2:172.21.15.149:6800/2978707630 boot 2024-09-10T18:59:32.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:31 smithi047 ceph-mon[39856]: osdmap e31: 5 total, 5 up, 5 in 2024-09-10T18:59:32.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:31 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:59:32.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:31 smithi149 ceph-mon[33230]: purged_snaps scrub starts 2024-09-10T18:59:32.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:31 smithi149 ceph-mon[33230]: purged_snaps scrub ok 2024-09-10T18:59:32.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:31 smithi149 ceph-mon[33230]: osd.4 v2:172.21.15.149:6800/2978707630 boot 2024-09-10T18:59:32.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:31 smithi149 ceph-mon[33230]: osdmap e31: 5 total, 5 up, 5 in 2024-09-10T18:59:32.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:31 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:59:32.685 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8563d7e-6fa5-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi149:vg_nvme/lv_3 2024-09-10T18:59:32.927 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:59:33.254 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:33 smithi149 ceph-mon[33230]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:59:33.254 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:33 smithi149 ceph-mon[33230]: osdmap e32: 5 total, 5 up, 5 in 2024-09-10T18:59:33.254 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:33 smithi149 ceph-mon[33230]: Detected new or changed devices on smithi149 2024-09-10T18:59:33.254 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:33 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:33.254 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:33 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:33.254 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:33 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:59:33.254 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:33 smithi149 ceph-mon[33230]: Adjusting osd_memory_target on smithi149 to 5815M 2024-09-10T18:59:33.254 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:33 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:33.254 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:33 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:59:33.254 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:33 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:59:33.255 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:33 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:33.289 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:33 smithi047 ceph-mon[39856]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:59:33.289 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:33 smithi047 ceph-mon[39856]: osdmap e32: 5 total, 5 up, 5 in 2024-09-10T18:59:33.289 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:33 smithi047 ceph-mon[39856]: Detected new or changed devices on smithi149 2024-09-10T18:59:33.289 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:33 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:33.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:33 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:33.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:33 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:59:33.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:33 smithi047 ceph-mon[39856]: Adjusting osd_memory_target on smithi149 to 5815M 2024-09-10T18:59:33.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:33 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:33.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:33 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:59:33.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:33 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:59:33.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:33 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:33.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:33 smithi047 ceph-mon[29019]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:59:33.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:33 smithi047 ceph-mon[29019]: osdmap e32: 5 total, 5 up, 5 in 2024-09-10T18:59:33.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:33 smithi047 ceph-mon[29019]: Detected new or changed devices on smithi149 2024-09-10T18:59:33.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:33 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:33.291 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:33 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:33.291 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:33 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:59:33.291 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:33 smithi047 ceph-mon[29019]: Adjusting osd_memory_target on smithi149 to 5815M 2024-09-10T18:59:33.291 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:33 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:33.291 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:33 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:59:33.291 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:33 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:59:33.291 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:33 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:34.138 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:34 smithi149 ceph-mon[33230]: osdmap e33: 5 total, 5 up, 5 in 2024-09-10T18:59:34.289 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:34 smithi047 ceph-mon[29019]: osdmap e33: 5 total, 5 up, 5 in 2024-09-10T18:59:34.289 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:34 smithi047 ceph-mon[39856]: osdmap e33: 5 total, 5 up, 5 in 2024-09-10T18:59:35.190 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:35 smithi149 ceph-mon[33230]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:59:35.190 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:35 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:59:35.190 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:35 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:59:35.190 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:35 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:59:35.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:35 smithi047 ceph-mon[29019]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:59:35.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:35 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:59:35.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:35 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:59:35.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:35 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:59:35.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:35 smithi047 ceph-mon[39856]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:59:35.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:35 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:59:35.291 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:35 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:59:35.291 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:35 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:59:36.289 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:36 smithi047 ceph-mon[29019]: from='client.24214 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:59:36.289 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:36 smithi047 ceph-mon[39856]: from='client.24214 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:59:36.291 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:36 smithi149 ceph-mon[33230]: from='client.24214 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:59:37.087 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:37 smithi149 ceph-mon[33230]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T18:59:37.087 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:37 smithi149 ceph-mon[33230]: from='client.? 172.21.15.149:0/480643654' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8ad2f515-f818-40d5-9318-3fb5c932c740"}]: dispatch 2024-09-10T18:59:37.088 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:37 smithi149 ceph-mon[33230]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8ad2f515-f818-40d5-9318-3fb5c932c740"}]: dispatch 2024-09-10T18:59:37.088 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:37 smithi149 ceph-mon[33230]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8ad2f515-f818-40d5-9318-3fb5c932c740"}]': finished 2024-09-10T18:59:37.088 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:37 smithi149 ceph-mon[33230]: osdmap e34: 6 total, 5 up, 6 in 2024-09-10T18:59:37.088 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:37 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:59:37.289 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:37 smithi047 ceph-mon[29019]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T18:59:37.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:37 smithi047 ceph-mon[29019]: from='client.? 172.21.15.149:0/480643654' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8ad2f515-f818-40d5-9318-3fb5c932c740"}]: dispatch 2024-09-10T18:59:37.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:37 smithi047 ceph-mon[29019]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8ad2f515-f818-40d5-9318-3fb5c932c740"}]: dispatch 2024-09-10T18:59:37.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:37 smithi047 ceph-mon[29019]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8ad2f515-f818-40d5-9318-3fb5c932c740"}]': finished 2024-09-10T18:59:37.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:37 smithi047 ceph-mon[29019]: osdmap e34: 6 total, 5 up, 6 in 2024-09-10T18:59:37.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:37 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:59:37.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:37 smithi047 ceph-mon[39856]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T18:59:37.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:37 smithi047 ceph-mon[39856]: from='client.? 172.21.15.149:0/480643654' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8ad2f515-f818-40d5-9318-3fb5c932c740"}]: dispatch 2024-09-10T18:59:37.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:37 smithi047 ceph-mon[39856]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8ad2f515-f818-40d5-9318-3fb5c932c740"}]: dispatch 2024-09-10T18:59:37.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:37 smithi047 ceph-mon[39856]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8ad2f515-f818-40d5-9318-3fb5c932c740"}]': finished 2024-09-10T18:59:37.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:37 smithi047 ceph-mon[39856]: osdmap e34: 6 total, 5 up, 6 in 2024-09-10T18:59:37.291 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:37 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:59:38.292 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:38 smithi047 ceph-mon[29019]: from='client.? 172.21.15.149:0/1975525937' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:59:38.292 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:38 smithi047 ceph-mon[39856]: from='client.? 172.21.15.149:0/1975525937' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:59:38.342 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:38 smithi149 ceph-mon[33230]: from='client.? 172.21.15.149:0/1975525937' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:59:39.288 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:39 smithi149 ceph-mon[33230]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T18:59:39.289 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:39 smithi047 ceph-mon[29019]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T18:59:39.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:39 smithi047 ceph-mon[39856]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T18:59:41.090 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:41 smithi149 ceph-mon[33230]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T18:59:41.289 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:41 smithi047 ceph-mon[29019]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T18:59:41.289 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:41 smithi047 ceph-mon[39856]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T18:59:43.219 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:43 smithi149 ceph-mon[33230]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-09-10T18:59:43.219 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:43 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-10T18:59:43.219 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:43 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:59:43.220 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:43 smithi149 ceph-mon[33230]: Deploying daemon osd.5 on smithi149 2024-09-10T18:59:43.289 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:43 smithi047 ceph-mon[29019]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-09-10T18:59:43.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:43 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-10T18:59:43.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:43 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:59:43.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:43 smithi047 ceph-mon[29019]: Deploying daemon osd.5 on smithi149 2024-09-10T18:59:43.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:43 smithi047 ceph-mon[39856]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-09-10T18:59:43.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:43 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-10T18:59:43.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:43 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:59:43.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:43 smithi047 ceph-mon[39856]: Deploying daemon osd.5 on smithi149 2024-09-10T18:59:45.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:45 smithi149 ceph-mon[33230]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T18:59:45.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:45 smithi047 ceph-mon[29019]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T18:59:45.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:45 smithi047 ceph-mon[39856]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T18:59:47.289 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:47 smithi047 ceph-mon[29019]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:59:47.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:47 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:59:47.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:47 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:47.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:47 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:47.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:47 smithi047 ceph-mon[39856]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:59:47.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:47 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:59:47.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:47 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:47.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:47 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:47.324 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:47 smithi149 ceph-mon[33230]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:59:47.324 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:47 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:59:47.324 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:47 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:47.325 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:47 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:49.223 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:49 smithi149 ceph-mon[33230]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:59:49.289 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:49 smithi047 ceph-mon[29019]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:59:49.289 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:49 smithi047 ceph-mon[39856]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:59:50.088 INFO:teuthology.orchestra.run.smithi149.stdout:Created osd(s) 5 on host 'smithi149' 2024-09-10T18:59:50.647 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:50 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:50.647 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:50 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:50.647 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:50 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:59:50.647 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:50 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:59:50.648 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:50 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:50.648 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:50 smithi149 ceph-mon[33230]: from='osd.5 v2:172.21.15.149:6804/902941407' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T18:59:50.648 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:50 smithi149 ceph-mon[33230]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T18:59:50.648 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:50 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:59:50.648 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:50 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:50.648 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:50 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:50.671 DEBUG:teuthology.orchestra.run.smithi149:osd.5> sudo journalctl -f -n 0 -u ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968@osd.5.service 2024-09-10T18:59:50.674 INFO:tasks.cephadm:Deploying osd.6 on smithi149 with /dev/vg_nvme/lv_2... 2024-09-10T18:59:50.674 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8563d7e-6fa5-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-09-10T18:59:50.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:50 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:50.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:50 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:50.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:50 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:59:50.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:50 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:59:50.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:50 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:50.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:50 smithi047 ceph-mon[29019]: from='osd.5 v2:172.21.15.149:6804/902941407' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T18:59:50.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:50 smithi047 ceph-mon[29019]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T18:59:50.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:50 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:59:50.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:50 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:50.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:50 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:50.791 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:50 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:50.791 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:50 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:50.791 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:50 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:59:50.791 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:50 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:59:50.791 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:50 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:50.791 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:50 smithi047 ceph-mon[39856]: from='osd.5 v2:172.21.15.149:6804/902941407' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T18:59:50.791 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:50 smithi047 ceph-mon[39856]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T18:59:50.791 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:50 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:59:50.791 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:50 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:50.791 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:50 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:51.663 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:51 smithi149 ceph-mon[33230]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:59:51.664 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:51 smithi149 ceph-mon[33230]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-10T18:59:51.664 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:51 smithi149 ceph-mon[33230]: osdmap e35: 6 total, 5 up, 6 in 2024-09-10T18:59:51.664 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:51 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:59:51.664 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:51 smithi149 ceph-mon[33230]: from='osd.5 v2:172.21.15.149:6804/902941407' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-09-10T18:59:51.664 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:51 smithi149 ceph-mon[33230]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-09-10T18:59:51.664 INFO:journalctl@ceph.osd.5.smithi149.stdout:Sep 10 18:59:51 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-5[44040]: 2024-09-10T18:59:51.403+0000 7fd4aad52640 -1 osd.5 0 waiting for initial osdmap 2024-09-10T18:59:51.664 INFO:journalctl@ceph.osd.5.smithi149.stdout:Sep 10 18:59:51 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-5[44040]: 2024-09-10T18:59:51.407+0000 7fd4a638a640 -1 osd.5 36 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T18:59:51.722 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:59:51.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:51 smithi047 ceph-mon[29019]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:59:51.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:51 smithi047 ceph-mon[29019]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-10T18:59:51.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:51 smithi047 ceph-mon[29019]: osdmap e35: 6 total, 5 up, 6 in 2024-09-10T18:59:51.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:51 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:59:51.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:51 smithi047 ceph-mon[29019]: from='osd.5 v2:172.21.15.149:6804/902941407' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-09-10T18:59:51.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:51 smithi047 ceph-mon[29019]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-09-10T18:59:51.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:51 smithi047 ceph-mon[39856]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:59:51.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:51 smithi047 ceph-mon[39856]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-10T18:59:51.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:51 smithi047 ceph-mon[39856]: osdmap e35: 6 total, 5 up, 6 in 2024-09-10T18:59:51.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:51 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:59:51.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:51 smithi047 ceph-mon[39856]: from='osd.5 v2:172.21.15.149:6804/902941407' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-09-10T18:59:51.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:51 smithi047 ceph-mon[39856]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-09-10T18:59:52.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:52 smithi149 ceph-mon[33230]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-09-10T18:59:52.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:52 smithi149 ceph-mon[33230]: osdmap e36: 6 total, 5 up, 6 in 2024-09-10T18:59:52.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:52 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:59:52.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:52 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:59:52.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:52 smithi149 ceph-mon[33230]: pgmap v123: 1 pgs: 1 peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:59:52.789 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:52 smithi047 ceph-mon[39856]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-09-10T18:59:52.789 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:52 smithi047 ceph-mon[39856]: osdmap e36: 6 total, 5 up, 6 in 2024-09-10T18:59:52.789 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:52 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:59:52.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:52 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:59:52.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:52 smithi047 ceph-mon[39856]: pgmap v123: 1 pgs: 1 peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:59:52.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:52 smithi047 ceph-mon[29019]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-09-10T18:59:52.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:52 smithi047 ceph-mon[29019]: osdmap e36: 6 total, 5 up, 6 in 2024-09-10T18:59:52.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:52 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:59:52.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:52 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:59:52.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:52 smithi047 ceph-mon[29019]: pgmap v123: 1 pgs: 1 peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:59:53.592 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:53 smithi149 ceph-mon[33230]: purged_snaps scrub starts 2024-09-10T18:59:53.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:53 smithi149 ceph-mon[33230]: purged_snaps scrub ok 2024-09-10T18:59:53.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:53 smithi149 ceph-mon[33230]: osd.5 v2:172.21.15.149:6804/902941407 boot 2024-09-10T18:59:53.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:53 smithi149 ceph-mon[33230]: osdmap e37: 6 total, 6 up, 6 in 2024-09-10T18:59:53.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:53 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:59:53.789 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:53 smithi047 ceph-mon[39856]: purged_snaps scrub starts 2024-09-10T18:59:53.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:53 smithi047 ceph-mon[39856]: purged_snaps scrub ok 2024-09-10T18:59:53.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:53 smithi047 ceph-mon[39856]: osd.5 v2:172.21.15.149:6804/902941407 boot 2024-09-10T18:59:53.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:53 smithi047 ceph-mon[39856]: osdmap e37: 6 total, 6 up, 6 in 2024-09-10T18:59:53.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:53 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:59:53.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:53 smithi047 ceph-mon[29019]: purged_snaps scrub starts 2024-09-10T18:59:53.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:53 smithi047 ceph-mon[29019]: purged_snaps scrub ok 2024-09-10T18:59:53.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:53 smithi047 ceph-mon[29019]: osd.5 v2:172.21.15.149:6804/902941407 boot 2024-09-10T18:59:53.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:53 smithi047 ceph-mon[29019]: osdmap e37: 6 total, 6 up, 6 in 2024-09-10T18:59:53.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:53 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:59:54.492 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-09-10T18:59:54.586 INFO:teuthology.orchestra.run.smithi149.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-09-10T18:59:54.586 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:54 smithi149 ceph-mon[33230]: osdmap e38: 6 total, 6 up, 6 in 2024-09-10T18:59:54.586 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:54 smithi149 ceph-mon[33230]: Detected new or changed devices on smithi149 2024-09-10T18:59:54.586 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:54 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:54.586 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:54 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:54.586 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:54 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:59:54.587 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:54 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:59:54.587 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:54 smithi149 ceph-mon[33230]: Adjusting osd_memory_target on smithi149 to 2907M 2024-09-10T18:59:54.587 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:54 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:54.587 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:54 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:59:54.587 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:54 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:59:54.587 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:54 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:54.587 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:54 smithi149 ceph-mon[33230]: pgmap v126: 1 pgs: 1 peering; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:59:54.601 INFO:teuthology.orchestra.run.smithi149.stderr: stderr: 10+0 records in 2024-09-10T18:59:54.602 INFO:teuthology.orchestra.run.smithi149.stderr:10+0 records out 2024-09-10T18:59:54.602 INFO:teuthology.orchestra.run.smithi149.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.01329 s, 789 MB/s 2024-09-10T18:59:54.602 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping successful for: 2024-09-10T18:59:54.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:54 smithi047 ceph-mon[29019]: osdmap e38: 6 total, 6 up, 6 in 2024-09-10T18:59:54.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:54 smithi047 ceph-mon[29019]: Detected new or changed devices on smithi149 2024-09-10T18:59:54.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:54 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:54.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:54 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:54.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:54 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:59:54.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:54 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:59:54.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:54 smithi047 ceph-mon[29019]: Adjusting osd_memory_target on smithi149 to 2907M 2024-09-10T18:59:54.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:54 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:54.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:54 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:59:54.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:54 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:59:54.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:54 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:54.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:54 smithi047 ceph-mon[29019]: pgmap v126: 1 pgs: 1 peering; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:59:54.791 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:54 smithi047 ceph-mon[39856]: osdmap e38: 6 total, 6 up, 6 in 2024-09-10T18:59:54.791 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:54 smithi047 ceph-mon[39856]: Detected new or changed devices on smithi149 2024-09-10T18:59:54.791 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:54 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:54.791 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:54 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:54.791 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:54 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:59:54.791 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:54 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:59:54.791 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:54 smithi047 ceph-mon[39856]: Adjusting osd_memory_target on smithi149 to 2907M 2024-09-10T18:59:54.791 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:54 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:54.791 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:54 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:59:54.791 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:54 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:59:54.791 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:54 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T18:59:54.791 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:54 smithi047 ceph-mon[39856]: pgmap v126: 1 pgs: 1 peering; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:59:55.193 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8563d7e-6fa5-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi149:vg_nvme/lv_2 2024-09-10T18:59:55.435 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:59:55.550 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:55 smithi149 ceph-mon[33230]: osdmap e39: 6 total, 6 up, 6 in 2024-09-10T18:59:55.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:55 smithi047 ceph-mon[29019]: osdmap e39: 6 total, 6 up, 6 in 2024-09-10T18:59:55.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:55 smithi047 ceph-mon[39856]: osdmap e39: 6 total, 6 up, 6 in 2024-09-10T18:59:56.430 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:56 smithi149 ceph-mon[33230]: pgmap v128: 1 pgs: 1 peering; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:59:56.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:56 smithi047 ceph-mon[29019]: pgmap v128: 1 pgs: 1 peering; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:59:56.789 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:56 smithi047 ceph-mon[39856]: pgmap v128: 1 pgs: 1 peering; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:59:57.702 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:57 smithi149 ceph-mon[33230]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-10T18:59:57.702 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:57 smithi149 ceph-mon[33230]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:59:57.702 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:57 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:59:57.702 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:57 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:59:57.702 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:57 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:59:57.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:57 smithi047 ceph-mon[29019]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-10T18:59:57.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:57 smithi047 ceph-mon[29019]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:59:57.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:57 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:59:57.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:57 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:59:57.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:57 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:59:57.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:57 smithi047 ceph-mon[39856]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-10T18:59:57.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:57 smithi047 ceph-mon[39856]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:59:57.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:57 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:59:57.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:57 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:59:57.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:57 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:59:58.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:58 smithi149 ceph-mon[33230]: pgmap v129: 1 pgs: 1 peering; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:59:58.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:58 smithi047 ceph-mon[29019]: pgmap v129: 1 pgs: 1 peering; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:59:58.789 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:58 smithi047 ceph-mon[39856]: pgmap v129: 1 pgs: 1 peering; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:59:59.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:59 smithi047 ceph-mon[29019]: from='client.? 172.21.15.149:0/3230421000' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d7421e61-3716-41c2-862d-8a91c91888e9"}]: dispatch 2024-09-10T18:59:59.929 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:59 smithi047 ceph-mon[29019]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d7421e61-3716-41c2-862d-8a91c91888e9"}]: dispatch 2024-09-10T18:59:59.929 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:59 smithi047 ceph-mon[29019]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d7421e61-3716-41c2-862d-8a91c91888e9"}]': finished 2024-09-10T18:59:59.930 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:59 smithi047 ceph-mon[29019]: osdmap e40: 7 total, 6 up, 7 in 2024-09-10T18:59:59.930 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 18:59:59 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:59:59.930 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:59 smithi047 ceph-mon[39856]: from='client.? 172.21.15.149:0/3230421000' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d7421e61-3716-41c2-862d-8a91c91888e9"}]: dispatch 2024-09-10T18:59:59.930 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:59 smithi047 ceph-mon[39856]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d7421e61-3716-41c2-862d-8a91c91888e9"}]: dispatch 2024-09-10T18:59:59.930 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:59 smithi047 ceph-mon[39856]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d7421e61-3716-41c2-862d-8a91c91888e9"}]': finished 2024-09-10T18:59:59.930 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:59 smithi047 ceph-mon[39856]: osdmap e40: 7 total, 6 up, 7 in 2024-09-10T18:59:59.930 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 18:59:59 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:59:59.931 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:59 smithi149 ceph-mon[33230]: from='client.? 172.21.15.149:0/3230421000' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d7421e61-3716-41c2-862d-8a91c91888e9"}]: dispatch 2024-09-10T18:59:59.931 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:59 smithi149 ceph-mon[33230]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d7421e61-3716-41c2-862d-8a91c91888e9"}]: dispatch 2024-09-10T18:59:59.931 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:59 smithi149 ceph-mon[33230]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d7421e61-3716-41c2-862d-8a91c91888e9"}]': finished 2024-09-10T18:59:59.931 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:59 smithi149 ceph-mon[33230]: osdmap e40: 7 total, 6 up, 7 in 2024-09-10T18:59:59.931 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 18:59:59 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T19:00:00.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:00 smithi149 ceph-mon[33230]: from='client.? 172.21.15.149:0/3638669882' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T19:00:00.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:00 smithi149 ceph-mon[33230]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-09-10T19:00:00.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:00 smithi149 ceph-mon[33230]: Health detail: HEALTH_WARN Reduced data availability: 1 pg peering 2024-09-10T19:00:00.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:00 smithi149 ceph-mon[33230]: [WRN] PG_AVAILABILITY: Reduced data availability: 1 pg peering 2024-09-10T19:00:00.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:00 smithi149 ceph-mon[33230]: pg 1.0 is stuck peering for 63s, current state peering, last acting [0,1] 2024-09-10T19:00:00.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:00 smithi047 ceph-mon[29019]: from='client.? 172.21.15.149:0/3638669882' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T19:00:00.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:00 smithi047 ceph-mon[29019]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-09-10T19:00:00.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:00 smithi047 ceph-mon[29019]: Health detail: HEALTH_WARN Reduced data availability: 1 pg peering 2024-09-10T19:00:00.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:00 smithi047 ceph-mon[29019]: [WRN] PG_AVAILABILITY: Reduced data availability: 1 pg peering 2024-09-10T19:00:00.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:00 smithi047 ceph-mon[29019]: pg 1.0 is stuck peering for 63s, current state peering, last acting [0,1] 2024-09-10T19:00:00.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:00 smithi047 ceph-mon[39856]: from='client.? 172.21.15.149:0/3638669882' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T19:00:00.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:00 smithi047 ceph-mon[39856]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-09-10T19:00:00.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:00 smithi047 ceph-mon[39856]: Health detail: HEALTH_WARN Reduced data availability: 1 pg peering 2024-09-10T19:00:00.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:00 smithi047 ceph-mon[39856]: [WRN] PG_AVAILABILITY: Reduced data availability: 1 pg peering 2024-09-10T19:00:00.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:00 smithi047 ceph-mon[39856]: pg 1.0 is stuck peering for 63s, current state peering, last acting [0,1] 2024-09-10T19:00:01.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:01 smithi047 ceph-mon[29019]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-10T19:00:01.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:01 smithi047 ceph-mon[29019]: Cluster is now healthy 2024-09-10T19:00:01.789 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:01 smithi047 ceph-mon[39856]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-10T19:00:01.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:01 smithi047 ceph-mon[39856]: Cluster is now healthy 2024-09-10T19:00:01.842 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:01 smithi149 ceph-mon[33230]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-10T19:00:01.843 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:01 smithi149 ceph-mon[33230]: Cluster is now healthy 2024-09-10T19:00:02.789 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:02 smithi047 ceph-mon[39856]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T19:00:02.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:02 smithi047 ceph-mon[29019]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T19:00:02.842 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:02 smithi149 ceph-mon[33230]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T19:00:05.092 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:05 smithi149 ceph-mon[33230]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-10T19:00:05.289 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:05 smithi047 ceph-mon[29019]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-10T19:00:05.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:05 smithi047 ceph-mon[39856]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-10T19:00:06.289 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:06 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-10T19:00:06.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:06 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:00:06.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:06 smithi047 ceph-mon[39856]: Deploying daemon osd.6 on smithi149 2024-09-10T19:00:06.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:06 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-10T19:00:06.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:06 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:00:06.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:06 smithi047 ceph-mon[29019]: Deploying daemon osd.6 on smithi149 2024-09-10T19:00:06.342 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:06 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-10T19:00:06.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:06 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:00:06.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:06 smithi149 ceph-mon[33230]: Deploying daemon osd.6 on smithi149 2024-09-10T19:00:07.289 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:07 smithi047 ceph-mon[29019]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T19:00:07.289 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:07 smithi047 ceph-mon[39856]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T19:00:07.338 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:07 smithi149 ceph-mon[33230]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T19:00:09.087 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:09 smithi149 ceph-mon[33230]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T19:00:09.289 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:09 smithi047 ceph-mon[39856]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T19:00:09.289 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:09 smithi047 ceph-mon[29019]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T19:00:10.289 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:10 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:00:10.289 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:10 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:10.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:10 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:10.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:10 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:00:10.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:10 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:10.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:10 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:10.325 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:10 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:00:10.325 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:10 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:10.325 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:10 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:11.289 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:11 smithi149 ceph-mon[33230]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T19:00:11.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:11 smithi047 ceph-mon[29019]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T19:00:11.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:11 smithi047 ceph-mon[39856]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T19:00:12.819 INFO:teuthology.orchestra.run.smithi149.stdout:Created osd(s) 6 on host 'smithi149' 2024-09-10T19:00:13.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:13 smithi149 ceph-mon[33230]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T19:00:13.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:13 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:13.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:13 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:13.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:13 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:00:13.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:13 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:00:13.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:13 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:13.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:13 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:00:13.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:13 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:13.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:13 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:13.344 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:13 smithi149 ceph-mon[33230]: from='osd.6 v2:172.21.15.149:6808/525682869' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-10T19:00:13.344 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:13 smithi149 ceph-mon[33230]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-10T19:00:13.492 DEBUG:teuthology.orchestra.run.smithi149:osd.6> sudo journalctl -f -n 0 -u ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968@osd.6.service 2024-09-10T19:00:13.494 INFO:tasks.cephadm:Deploying osd.7 on smithi149 with /dev/vg_nvme/lv_1... 2024-09-10T19:00:13.494 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8563d7e-6fa5-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-09-10T19:00:13.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:13 smithi047 ceph-mon[29019]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T19:00:13.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:13 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:13.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:13 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:13.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:13 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:00:13.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:13 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:00:13.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:13 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:13.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:13 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:00:13.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:13 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:13.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:13 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:13.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:13 smithi047 ceph-mon[29019]: from='osd.6 v2:172.21.15.149:6808/525682869' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-10T19:00:13.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:13 smithi047 ceph-mon[29019]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-10T19:00:13.541 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:13 smithi047 ceph-mon[39856]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T19:00:13.541 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:13 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:13.541 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:13 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:13.541 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:13 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:00:13.541 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:13 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:00:13.541 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:13 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:13.541 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:13 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:00:13.541 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:13 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:13.541 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:13 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:13.541 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:13 smithi047 ceph-mon[39856]: from='osd.6 v2:172.21.15.149:6808/525682869' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-10T19:00:13.541 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:13 smithi047 ceph-mon[39856]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-10T19:00:14.830 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:14 smithi149 ceph-mon[33230]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-10T19:00:14.830 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:14 smithi149 ceph-mon[33230]: osdmap e41: 7 total, 6 up, 7 in 2024-09-10T19:00:14.830 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:14 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T19:00:14.830 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:14 smithi149 ceph-mon[33230]: from='osd.6 v2:172.21.15.149:6808/525682869' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-09-10T19:00:14.830 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:14 smithi149 ceph-mon[33230]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-09-10T19:00:14.830 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:14 smithi149 ceph-mon[33230]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T19:00:15.034 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T19:00:15.092 INFO:journalctl@ceph.osd.6.smithi149.stdout:Sep 10 19:00:14 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-6[49840]: 2024-09-10T19:00:14.827+0000 7f20c72d9640 -1 osd.6 0 waiting for initial osdmap 2024-09-10T19:00:15.092 INFO:journalctl@ceph.osd.6.smithi149.stdout:Sep 10 19:00:14 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-6[49840]: 2024-09-10T19:00:14.833+0000 7f20c4114640 -1 osd.6 42 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T19:00:15.289 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:14 smithi047 ceph-mon[29019]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-10T19:00:15.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:14 smithi047 ceph-mon[29019]: osdmap e41: 7 total, 6 up, 7 in 2024-09-10T19:00:15.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:14 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T19:00:15.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:14 smithi047 ceph-mon[29019]: from='osd.6 v2:172.21.15.149:6808/525682869' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-09-10T19:00:15.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:14 smithi047 ceph-mon[29019]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-09-10T19:00:15.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:14 smithi047 ceph-mon[29019]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T19:00:15.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:14 smithi047 ceph-mon[39856]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-10T19:00:15.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:14 smithi047 ceph-mon[39856]: osdmap e41: 7 total, 6 up, 7 in 2024-09-10T19:00:15.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:14 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T19:00:15.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:14 smithi047 ceph-mon[39856]: from='osd.6 v2:172.21.15.149:6808/525682869' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-09-10T19:00:15.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:14 smithi047 ceph-mon[39856]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-09-10T19:00:15.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:14 smithi047 ceph-mon[39856]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T19:00:16.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:15 smithi149 ceph-mon[33230]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-09-10T19:00:16.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:15 smithi149 ceph-mon[33230]: osdmap e42: 7 total, 6 up, 7 in 2024-09-10T19:00:16.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:15 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T19:00:16.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:15 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T19:00:16.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:15 smithi149 ceph-mon[33230]: osd.6 v2:172.21.15.149:6808/525682869 boot 2024-09-10T19:00:16.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:15 smithi149 ceph-mon[33230]: osdmap e43: 7 total, 7 up, 7 in 2024-09-10T19:00:16.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:15 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T19:00:16.289 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:15 smithi047 ceph-mon[29019]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-09-10T19:00:16.289 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:15 smithi047 ceph-mon[29019]: osdmap e42: 7 total, 6 up, 7 in 2024-09-10T19:00:16.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:15 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T19:00:16.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:15 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T19:00:16.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:15 smithi047 ceph-mon[29019]: osd.6 v2:172.21.15.149:6808/525682869 boot 2024-09-10T19:00:16.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:15 smithi047 ceph-mon[29019]: osdmap e43: 7 total, 7 up, 7 in 2024-09-10T19:00:16.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:15 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T19:00:16.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:15 smithi047 ceph-mon[39856]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-09-10T19:00:16.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:15 smithi047 ceph-mon[39856]: osdmap e42: 7 total, 6 up, 7 in 2024-09-10T19:00:16.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:15 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T19:00:16.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:15 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T19:00:16.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:15 smithi047 ceph-mon[39856]: osd.6 v2:172.21.15.149:6808/525682869 boot 2024-09-10T19:00:16.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:15 smithi047 ceph-mon[39856]: osdmap e43: 7 total, 7 up, 7 in 2024-09-10T19:00:16.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:15 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T19:00:16.950 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:16 smithi149 ceph-mon[33230]: purged_snaps scrub starts 2024-09-10T19:00:16.950 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:16 smithi149 ceph-mon[33230]: purged_snaps scrub ok 2024-09-10T19:00:16.950 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:16 smithi149 ceph-mon[33230]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T19:00:16.950 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:16 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:16.950 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:16 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:16.950 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:16 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:00:16.950 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:16 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:00:16.950 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:16 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:00:16.951 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:16 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:16.951 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:16 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:00:16.951 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:16 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:00:16.951 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:16 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:17.272 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-09-10T19:00:17.289 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:16 smithi047 ceph-mon[29019]: purged_snaps scrub starts 2024-09-10T19:00:17.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:16 smithi047 ceph-mon[29019]: purged_snaps scrub ok 2024-09-10T19:00:17.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:16 smithi047 ceph-mon[29019]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T19:00:17.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:16 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:17.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:16 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:17.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:16 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:00:17.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:16 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:00:17.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:16 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:00:17.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:16 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:17.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:16 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:00:17.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:16 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:00:17.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:16 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:17.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:16 smithi047 ceph-mon[39856]: purged_snaps scrub starts 2024-09-10T19:00:17.291 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:16 smithi047 ceph-mon[39856]: purged_snaps scrub ok 2024-09-10T19:00:17.291 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:16 smithi047 ceph-mon[39856]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T19:00:17.291 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:16 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:17.291 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:16 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:17.291 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:16 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:00:17.291 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:16 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:00:17.291 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:16 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:00:17.291 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:16 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:17.291 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:16 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:00:17.291 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:16 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:00:17.291 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:16 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:17.359 INFO:teuthology.orchestra.run.smithi149.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-09-10T19:00:17.374 INFO:teuthology.orchestra.run.smithi149.stderr: stderr: 10+0 records in 2024-09-10T19:00:17.374 INFO:teuthology.orchestra.run.smithi149.stderr:10+0 records out 2024-09-10T19:00:17.374 INFO:teuthology.orchestra.run.smithi149.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0123158 s, 851 MB/s 2024-09-10T19:00:17.374 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping successful for: 2024-09-10T19:00:17.944 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8563d7e-6fa5-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi149:vg_nvme/lv_1 2024-09-10T19:00:18.092 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:17 smithi149 ceph-mon[33230]: Detected new or changed devices on smithi149 2024-09-10T19:00:18.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:17 smithi149 ceph-mon[33230]: Adjusting osd_memory_target on smithi149 to 1938M 2024-09-10T19:00:18.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:17 smithi149 ceph-mon[33230]: osdmap e44: 7 total, 7 up, 7 in 2024-09-10T19:00:18.187 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T19:00:18.289 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:17 smithi047 ceph-mon[29019]: Detected new or changed devices on smithi149 2024-09-10T19:00:18.289 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:17 smithi047 ceph-mon[29019]: Adjusting osd_memory_target on smithi149 to 1938M 2024-09-10T19:00:18.289 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:17 smithi047 ceph-mon[29019]: osdmap e44: 7 total, 7 up, 7 in 2024-09-10T19:00:18.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:17 smithi047 ceph-mon[39856]: Detected new or changed devices on smithi149 2024-09-10T19:00:18.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:17 smithi047 ceph-mon[39856]: Adjusting osd_memory_target on smithi149 to 1938M 2024-09-10T19:00:18.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:17 smithi047 ceph-mon[39856]: osdmap e44: 7 total, 7 up, 7 in 2024-09-10T19:00:19.018 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:18 smithi149 ceph-mon[33230]: osdmap e45: 7 total, 7 up, 7 in 2024-09-10T19:00:19.018 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:18 smithi149 ceph-mon[33230]: pgmap v145: 1 pgs: 1 remapped+peering; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T19:00:19.289 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:18 smithi047 ceph-mon[29019]: osdmap e45: 7 total, 7 up, 7 in 2024-09-10T19:00:19.289 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:18 smithi047 ceph-mon[29019]: pgmap v145: 1 pgs: 1 remapped+peering; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T19:00:19.289 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:18 smithi047 ceph-mon[39856]: osdmap e45: 7 total, 7 up, 7 in 2024-09-10T19:00:19.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:18 smithi047 ceph-mon[39856]: pgmap v145: 1 pgs: 1 remapped+peering; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T19:00:20.092 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:19 smithi149 ceph-mon[33230]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-10T19:00:20.289 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:19 smithi047 ceph-mon[29019]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-10T19:00:20.289 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:19 smithi047 ceph-mon[39856]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-10T19:00:21.185 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:20 smithi149 ceph-mon[33230]: from='client.24268 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:00:21.185 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:20 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T19:00:21.185 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:20 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T19:00:21.186 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:20 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:00:21.186 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:20 smithi149 ceph-mon[33230]: pgmap v146: 1 pgs: 1 remapped+peering; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T19:00:21.289 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:20 smithi047 ceph-mon[29019]: from='client.24268 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:00:21.289 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:20 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T19:00:21.289 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:20 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T19:00:21.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:20 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:00:21.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:20 smithi047 ceph-mon[29019]: pgmap v146: 1 pgs: 1 remapped+peering; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T19:00:21.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:20 smithi047 ceph-mon[39856]: from='client.24268 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:00:21.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:20 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T19:00:21.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:20 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T19:00:21.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:20 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:00:21.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:20 smithi047 ceph-mon[39856]: pgmap v146: 1 pgs: 1 remapped+peering; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T19:00:22.331 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:22 smithi149 ceph-mon[33230]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-10T19:00:22.331 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:22 smithi149 ceph-mon[33230]: Cluster is now healthy 2024-09-10T19:00:22.331 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:22 smithi149 ceph-mon[33230]: from='client.? 172.21.15.149:0/1768215853' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ad252ba5-666c-46b1-982d-c7e9ffa2b848"}]: dispatch 2024-09-10T19:00:22.331 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:22 smithi149 ceph-mon[33230]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ad252ba5-666c-46b1-982d-c7e9ffa2b848"}]: dispatch 2024-09-10T19:00:22.332 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:22 smithi149 ceph-mon[33230]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ad252ba5-666c-46b1-982d-c7e9ffa2b848"}]': finished 2024-09-10T19:00:22.332 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:22 smithi149 ceph-mon[33230]: osdmap e46: 8 total, 7 up, 8 in 2024-09-10T19:00:22.332 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:22 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T19:00:22.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:22 smithi047 ceph-mon[29019]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-10T19:00:22.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:22 smithi047 ceph-mon[29019]: Cluster is now healthy 2024-09-10T19:00:22.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:22 smithi047 ceph-mon[29019]: from='client.? 172.21.15.149:0/1768215853' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ad252ba5-666c-46b1-982d-c7e9ffa2b848"}]: dispatch 2024-09-10T19:00:22.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:22 smithi047 ceph-mon[29019]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ad252ba5-666c-46b1-982d-c7e9ffa2b848"}]: dispatch 2024-09-10T19:00:22.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:22 smithi047 ceph-mon[29019]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ad252ba5-666c-46b1-982d-c7e9ffa2b848"}]': finished 2024-09-10T19:00:22.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:22 smithi047 ceph-mon[29019]: osdmap e46: 8 total, 7 up, 8 in 2024-09-10T19:00:22.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:22 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T19:00:22.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:22 smithi047 ceph-mon[39856]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-10T19:00:22.541 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:22 smithi047 ceph-mon[39856]: Cluster is now healthy 2024-09-10T19:00:22.541 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:22 smithi047 ceph-mon[39856]: from='client.? 172.21.15.149:0/1768215853' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ad252ba5-666c-46b1-982d-c7e9ffa2b848"}]: dispatch 2024-09-10T19:00:22.541 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:22 smithi047 ceph-mon[39856]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ad252ba5-666c-46b1-982d-c7e9ffa2b848"}]: dispatch 2024-09-10T19:00:22.541 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:22 smithi047 ceph-mon[39856]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ad252ba5-666c-46b1-982d-c7e9ffa2b848"}]': finished 2024-09-10T19:00:22.541 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:22 smithi047 ceph-mon[39856]: osdmap e46: 8 total, 7 up, 8 in 2024-09-10T19:00:22.541 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:22 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T19:00:23.342 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:23 smithi149 ceph-mon[33230]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T19:00:23.342 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:23 smithi149 ceph-mon[33230]: from='client.? 172.21.15.149:0/3664521477' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T19:00:23.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:23 smithi047 ceph-mon[29019]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T19:00:23.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:23 smithi047 ceph-mon[29019]: from='client.? 172.21.15.149:0/3664521477' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T19:00:23.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:23 smithi047 ceph-mon[39856]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T19:00:23.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:23 smithi047 ceph-mon[39856]: from='client.? 172.21.15.149:0/3664521477' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T19:00:25.342 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:25 smithi149 ceph-mon[33230]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-09-10T19:00:25.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:25 smithi047 ceph-mon[29019]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-09-10T19:00:25.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:25 smithi047 ceph-mon[39856]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-09-10T19:00:27.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:27 smithi149 ceph-mon[33230]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-09-10T19:00:27.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:27 smithi047 ceph-mon[29019]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-09-10T19:00:27.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:27 smithi047 ceph-mon[39856]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-09-10T19:00:28.342 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:28 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-10T19:00:28.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:28 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:00:28.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:28 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-10T19:00:28.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:28 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:00:28.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:28 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-10T19:00:28.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:28 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:00:29.342 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:29 smithi149 ceph-mon[33230]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T19:00:29.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:29 smithi149 ceph-mon[33230]: Deploying daemon osd.7 on smithi149 2024-09-10T19:00:29.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:29 smithi047 ceph-mon[29019]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T19:00:29.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:29 smithi047 ceph-mon[29019]: Deploying daemon osd.7 on smithi149 2024-09-10T19:00:29.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:29 smithi047 ceph-mon[39856]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T19:00:29.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:29 smithi047 ceph-mon[39856]: Deploying daemon osd.7 on smithi149 2024-09-10T19:00:31.304 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:31 smithi149 ceph-mon[33230]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T19:00:31.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:31 smithi047 ceph-mon[29019]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T19:00:31.539 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:31 smithi047 ceph-mon[39856]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T19:00:33.356 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:33 smithi149 ceph-mon[33230]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T19:00:33.356 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:33 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:00:33.356 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:33 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:33.356 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:33 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:33.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:33 smithi047 ceph-mon[29019]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T19:00:33.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:33 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:00:33.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:33 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:33.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:33 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:33.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:33 smithi047 ceph-mon[39856]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T19:00:33.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:33 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:00:33.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:33 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:33.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:33 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:35.342 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:35 smithi149 ceph-mon[33230]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T19:00:35.368 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:35 smithi047 ceph-mon[29019]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T19:00:35.369 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:35 smithi047 ceph-mon[39856]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T19:00:35.968 INFO:teuthology.orchestra.run.smithi149.stdout:Created osd(s) 7 on host 'smithi149' 2024-09-10T19:00:36.480 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:36 smithi149 ceph-mon[33230]: from='osd.7 v2:172.21.15.149:6812/3375530086' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T19:00:36.480 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:36 smithi149 ceph-mon[33230]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T19:00:36.481 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:36 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:36.481 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:36 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:36.506 DEBUG:teuthology.orchestra.run.smithi149:osd.7> sudo journalctl -f -n 0 -u ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968@osd.7.service 2024-09-10T19:00:36.508 INFO:tasks.cephadm:Waiting for 8 OSDs to come up... 2024-09-10T19:00:36.509 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8563d7e-6fa5-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-10T19:00:36.534 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:36 smithi047 ceph-mon[29019]: from='osd.7 v2:172.21.15.149:6812/3375530086' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T19:00:36.534 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:36 smithi047 ceph-mon[29019]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T19:00:36.534 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:36 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:36.534 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:36 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:36.534 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:36 smithi047 ceph-mon[39856]: from='osd.7 v2:172.21.15.149:6812/3375530086' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T19:00:36.534 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:36 smithi047 ceph-mon[39856]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T19:00:36.534 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:36 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:36.535 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:36 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:36.767 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:00:37.141 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:37 smithi047 ceph-mon[29019]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T19:00:37.141 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:37 smithi047 ceph-mon[29019]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-10T19:00:37.141 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:37 smithi047 ceph-mon[29019]: osdmap e47: 8 total, 7 up, 8 in 2024-09-10T19:00:37.141 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:37 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T19:00:37.141 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:37 smithi047 ceph-mon[29019]: from='osd.7 v2:172.21.15.149:6812/3375530086' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-09-10T19:00:37.141 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:37 smithi047 ceph-mon[29019]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-09-10T19:00:37.141 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:37 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:37.141 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:37 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:37.142 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:37 smithi047 ceph-mon[39856]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T19:00:37.142 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:37 smithi047 ceph-mon[39856]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-10T19:00:37.142 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:37 smithi047 ceph-mon[39856]: osdmap e47: 8 total, 7 up, 8 in 2024-09-10T19:00:37.142 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:37 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T19:00:37.142 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:37 smithi047 ceph-mon[39856]: from='osd.7 v2:172.21.15.149:6812/3375530086' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-09-10T19:00:37.142 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:37 smithi047 ceph-mon[39856]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-09-10T19:00:37.142 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:37 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:37.142 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:37 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:37.429 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:37 smithi149 ceph-mon[33230]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T19:00:37.430 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:37 smithi149 ceph-mon[33230]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-10T19:00:37.430 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:37 smithi149 ceph-mon[33230]: osdmap e47: 8 total, 7 up, 8 in 2024-09-10T19:00:37.430 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:37 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T19:00:37.430 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:37 smithi149 ceph-mon[33230]: from='osd.7 v2:172.21.15.149:6812/3375530086' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-09-10T19:00:37.430 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:37 smithi149 ceph-mon[33230]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-09-10T19:00:37.430 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:37 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:37.430 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:37 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:37.430 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:00:37 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7[55520]: 2024-09-10T19:00:37.111+0000 7f0430f44640 -1 osd.7 0 waiting for initial osdmap 2024-09-10T19:00:37.430 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:00:37 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7[55520]: 2024-09-10T19:00:37.117+0000 7f042bd7b640 -1 osd.7 48 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T19:00:38.219 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:38 smithi047 ceph-mon[29019]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-09-10T19:00:38.219 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:38 smithi047 ceph-mon[29019]: osdmap e48: 8 total, 7 up, 8 in 2024-09-10T19:00:38.219 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:38 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T19:00:38.219 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:38 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T19:00:38.220 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:38 smithi047 ceph-mon[39856]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-09-10T19:00:38.220 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:38 smithi047 ceph-mon[39856]: osdmap e48: 8 total, 7 up, 8 in 2024-09-10T19:00:38.220 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:38 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T19:00:38.220 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:38 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T19:00:38.370 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:38 smithi149 ceph-mon[33230]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-09-10T19:00:38.371 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:38 smithi149 ceph-mon[33230]: osdmap e48: 8 total, 7 up, 8 in 2024-09-10T19:00:38.371 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:38 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T19:00:38.371 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:38 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T19:00:38.589 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T19:00:39.268 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:39 smithi047 ceph-mon[29019]: purged_snaps scrub starts 2024-09-10T19:00:39.268 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:39 smithi047 ceph-mon[29019]: purged_snaps scrub ok 2024-09-10T19:00:39.268 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:39 smithi047 ceph-mon[29019]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T19:00:39.268 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:39 smithi047 ceph-mon[29019]: osd.7 v2:172.21.15.149:6812/3375530086 boot 2024-09-10T19:00:39.268 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:39 smithi047 ceph-mon[29019]: osdmap e49: 8 total, 8 up, 8 in 2024-09-10T19:00:39.269 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:39 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T19:00:39.269 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:39 smithi047 ceph-mon[29019]: osdmap e50: 8 total, 8 up, 8 in 2024-09-10T19:00:39.269 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:39 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/1888898931' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T19:00:39.269 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:39 smithi047 ceph-mon[39856]: purged_snaps scrub starts 2024-09-10T19:00:39.269 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:39 smithi047 ceph-mon[39856]: purged_snaps scrub ok 2024-09-10T19:00:39.269 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:39 smithi047 ceph-mon[39856]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T19:00:39.269 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:39 smithi047 ceph-mon[39856]: osd.7 v2:172.21.15.149:6812/3375530086 boot 2024-09-10T19:00:39.269 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:39 smithi047 ceph-mon[39856]: osdmap e49: 8 total, 8 up, 8 in 2024-09-10T19:00:39.269 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:39 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T19:00:39.269 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:39 smithi047 ceph-mon[39856]: osdmap e50: 8 total, 8 up, 8 in 2024-09-10T19:00:39.270 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:39 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/1888898931' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T19:00:39.297 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":50,"num_osds":8,"num_up_osds":8,"osd_up_since":1725994838,"num_in_osds":8,"osd_in_since":1725994822,"num_remapped_pgs":0} 2024-09-10T19:00:39.298 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid d8563d7e-6fa5-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T19:00:39.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:39 smithi149 ceph-mon[33230]: purged_snaps scrub starts 2024-09-10T19:00:39.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:39 smithi149 ceph-mon[33230]: purged_snaps scrub ok 2024-09-10T19:00:39.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:39 smithi149 ceph-mon[33230]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T19:00:39.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:39 smithi149 ceph-mon[33230]: osd.7 v2:172.21.15.149:6812/3375530086 boot 2024-09-10T19:00:39.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:39 smithi149 ceph-mon[33230]: osdmap e49: 8 total, 8 up, 8 in 2024-09-10T19:00:39.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:39 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T19:00:39.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:39 smithi149 ceph-mon[33230]: osdmap e50: 8 total, 8 up, 8 in 2024-09-10T19:00:39.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:39 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/1888898931' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T19:00:39.597 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:00:40.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:40 smithi047 ceph-mon[39856]: osdmap e51: 8 total, 8 up, 8 in 2024-09-10T19:00:40.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:40 smithi047 ceph-mon[39856]: Detected new or changed devices on smithi149 2024-09-10T19:00:40.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:40 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:40.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:40 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:40.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:40 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:00:40.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:40 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:00:40.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:40 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:00:40.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:40 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:00:40.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:40 smithi047 ceph-mon[39856]: Adjusting osd_memory_target on smithi149 to 1453M 2024-09-10T19:00:40.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:40 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:40.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:40 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:00:40.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:40 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:00:40.791 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:40 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:40.791 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:40 smithi047 ceph-mon[39856]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:00:40.791 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:40 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:00:40.791 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:40 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:00:40.791 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:40 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:00:40.791 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:40 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:40.791 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:40 smithi047 ceph-mon[29019]: osdmap e51: 8 total, 8 up, 8 in 2024-09-10T19:00:40.791 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:40 smithi047 ceph-mon[29019]: Detected new or changed devices on smithi149 2024-09-10T19:00:40.791 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:40 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:40.791 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:40 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:40.791 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:40 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:00:40.791 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:40 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:00:40.791 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:40 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:00:40.792 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:40 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:00:40.792 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:40 smithi047 ceph-mon[29019]: Adjusting osd_memory_target on smithi149 to 1453M 2024-09-10T19:00:40.792 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:40 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:40.792 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:40 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:00:40.792 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:40 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:00:40.792 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:40 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:40.792 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:40 smithi047 ceph-mon[29019]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:00:40.792 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:40 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:00:40.792 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:40 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:00:40.792 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:40 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:00:40.792 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:40 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:40.845 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:40 smithi149 ceph-mon[33230]: osdmap e51: 8 total, 8 up, 8 in 2024-09-10T19:00:40.845 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:40 smithi149 ceph-mon[33230]: Detected new or changed devices on smithi149 2024-09-10T19:00:40.845 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:40 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:40.846 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:40 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:40.846 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:40 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:00:40.846 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:40 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:00:40.846 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:40 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:00:40.846 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:40 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:00:40.846 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:40 smithi149 ceph-mon[33230]: Adjusting osd_memory_target on smithi149 to 1453M 2024-09-10T19:00:40.846 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:40 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:40.846 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:40 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:00:40.846 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:40 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:00:40.846 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:40 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:40.846 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:40 smithi149 ceph-mon[33230]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:00:40.846 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:40 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:00:40.846 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:40 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:00:40.846 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:40 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:00:40.847 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:40 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:41.500 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T19:00:41.500 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":51,"fsid":"d8563d7e-6fa5-11ef-bcea-c7b262605968","created":"2024-09-10T18:55:47.433961+0000","modified":"2024-09-10T19:00:39.499289+0000","last_up_change":"2024-09-10T19:00:38.105604+0000","last_in_change":"2024-09-10T19:00:22.059888+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T18:58:48.422291+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"b4576db6-5c73-4a0e-af6f-6aa568540249","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6801","nonce":840905999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":840905999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":840905999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6803","nonce":840905999}]},"public_addr":"172.21.15.47:6801/840905999","cluster_addr":"172.21.15.47:6802/840905999","heartbeat_back_addr":"172.21.15.47:6804/840905999","heartbeat_front_addr":"172.21.15.47:6803/840905999","state":["exists","up"]},{"osd":1,"uuid":"5ecc9667-1ec5-47d0-b8b0-53411fa13b0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6805","nonce":3075184801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":3075184801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":3075184801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6807","nonce":3075184801}]},"public_addr":"172.21.15.47:6805/3075184801","cluster_addr":"172.21.15.47:6806/3075184801","heartbeat_back_addr":"172.21.15.47:6808/3075184801","heartbeat_front_addr":"172.21.15.47:6807/3075184801","state":["exists","up"]},{"osd":2,"uuid":"68a8c2ac-958c-411c-8a8f-f9e0b453901d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6809","nonce":1234745743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":1234745743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":1234745743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6811","nonce":1234745743}]},"public_addr":"172.21.15.47:6809/1234745743","cluster_addr":"172.21.15.47:6810/1234745743","heartbeat_back_addr":"172.21.15.47:6812/1234745743","heartbeat_front_addr":"172.21.15.47:6811/1234745743","state":["exists","up"]},{"osd":3,"uuid":"126cbc1e-5b4f-439f-9d03-49018692c2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6813","nonce":2685917187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":2685917187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2685917187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6815","nonce":2685917187}]},"public_addr":"172.21.15.47:6813/2685917187","cluster_addr":"172.21.15.47:6814/2685917187","heartbeat_back_addr":"172.21.15.47:6816/2685917187","heartbeat_front_addr":"172.21.15.47:6815/2685917187","state":["exists","up"]},{"osd":4,"uuid":"4a83f8dc-b265-4f60-a8e1-8bcc6fd4130c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":2978707630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6801","nonce":2978707630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6803","nonce":2978707630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":2978707630}]},"public_addr":"172.21.15.149:6800/2978707630","cluster_addr":"172.21.15.149:6801/2978707630","heartbeat_back_addr":"172.21.15.149:6803/2978707630","heartbeat_front_addr":"172.21.15.149:6802/2978707630","state":["exists","up"]},{"osd":5,"uuid":"8ad2f515-f818-40d5-9318-3fb5c932c740","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":902941407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6805","nonce":902941407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6807","nonce":902941407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":902941407}]},"public_addr":"172.21.15.149:6804/902941407","cluster_addr":"172.21.15.149:6805/902941407","heartbeat_back_addr":"172.21.15.149:6807/902941407","heartbeat_front_addr":"172.21.15.149:6806/902941407","state":["exists","up"]},{"osd":6,"uuid":"d7421e61-3716-41c2-862d-8a91c91888e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":525682869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6809","nonce":525682869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6811","nonce":525682869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":525682869}]},"public_addr":"172.21.15.149:6808/525682869","cluster_addr":"172.21.15.149:6809/525682869","heartbeat_back_addr":"172.21.15.149:6811/525682869","heartbeat_front_addr":"172.21.15.149:6810/525682869","state":["exists","up"]},{"osd":7,"uuid":"ad252ba5-666c-46b1-982d-c7e9ffa2b848","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":3375530086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6813","nonce":3375530086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6815","nonce":3375530086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3375530086}]},"public_addr":"172.21.15.149:6812/3375530086","cluster_addr":"172.21.15.149:6813/3375530086","heartbeat_back_addr":"172.21.15.149:6815/3375530086","heartbeat_front_addr":"172.21.15.149:6814/3375530086","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:57:59.034145+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:58:21.972178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:58:44.772844+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:59:07.565542+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:59:28.970709+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:59:50.869973+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T19:00:13.964897+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T19:00:36.809518+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:0/2845803815":"2024-09-11T18:56:37.929313+0000","172.21.15.47:6800/3835757035":"2024-09-11T18:56:15.562858+0000","172.21.15.47:6800/1465212466":"2024-09-11T18:56:37.929313+0000","172.21.15.47:0/351205668":"2024-09-11T18:56:37.929313+0000","172.21.15.47:0/1493757638":"2024-09-11T18:56:15.562858+0000","172.21.15.47:0/438725555":"2024-09-11T18:56:37.929313+0000","172.21.15.47:0/1917819301":"2024-09-11T18:56:15.562858+0000","172.21.15.47:0/1458078728":"2024-09-11T18:56:15.562858+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T19:00:42.107 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:41 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/2710509543' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T19:00:42.107 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:41 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/2710509543' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T19:00:42.132 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T18:58:48.422291+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2024-09-10T19:00:42.133 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid d8563d7e-6fa5-11ef-bcea-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-10T19:00:42.342 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:41 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/2710509543' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T19:00:42.383 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:00:43.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:42 smithi047 ceph-mon[29019]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:00:43.215 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:42 smithi047 ceph-mon[39856]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:00:43.342 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:42 smithi149 ceph-mon[33230]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:00:44.284 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-09-10T19:00:44.898 INFO:tasks.cephadm:Adding ceph.rgw.foo.a on smithi047 2024-09-10T19:00:44.899 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8563d7e-6fa5-11ef-bcea-c7b262605968 -- ceph orch apply rgw foo.a --placement '1;smithi047=foo.a' 2024-09-10T19:00:45.154 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T19:00:45.261 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:45 smithi149 ceph-mon[33230]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:00:45.261 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:45 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/345673467' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T19:00:45.289 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:45 smithi047 ceph-mon[39856]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:00:45.289 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:45 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/345673467' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T19:00:45.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:45 smithi047 ceph-mon[29019]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:00:45.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:45 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/345673467' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T19:00:46.854 INFO:teuthology.orchestra.run.smithi149.stdout:Scheduled rgw.foo.a update... 2024-09-10T19:00:47.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:47 smithi047 ceph-mon[39856]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:00:47.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:47 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:47.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:47 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:00:47.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:47 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:00:47.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:47 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:00:47.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:47 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:47.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:47 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-09-10T19:00:47.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:47 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-10T19:00:47.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:47 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:47.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:47 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:00:47.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:47 smithi047 ceph-mon[29019]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:00:47.291 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:47 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:47.291 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:47 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:00:47.291 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:47 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:00:47.291 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:47 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:00:47.291 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:47 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:47.291 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:47 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-09-10T19:00:47.291 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:47 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-10T19:00:47.291 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:47 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:47.291 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:47 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:00:47.342 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:47 smithi149 ceph-mon[33230]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:00:47.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:47 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:47.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:47 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:00:47.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:47 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:00:47.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:47 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:00:47.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:47 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:47.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:47 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-09-10T19:00:47.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:47 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-10T19:00:47.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:47 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:47.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:47 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:00:47.448 DEBUG:teuthology.orchestra.run.smithi047:rgw.foo.a> sudo journalctl -f -n 0 -u ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968@rgw.foo.a.service 2024-09-10T19:00:47.450 INFO:tasks.cephadm:Adding ceph.iscsi.iscsi.a on smithi149 2024-09-10T19:00:47.451 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8563d7e-6fa5-11ef-bcea-c7b262605968 -- ceph osd pool create datapool 3 3 replicated 2024-09-10T19:00:47.706 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T19:00:48.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:48 smithi149 ceph-mon[33230]: from='client.24304 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi047=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:00:48.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:48 smithi149 ceph-mon[33230]: Saving service rgw.foo.a spec with placement smithi047=foo.a;count:1 2024-09-10T19:00:48.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:48 smithi149 ceph-mon[33230]: Deploying daemon rgw.foo.a on smithi047 2024-09-10T19:00:48.446 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:48 smithi047 ceph-mon[39856]: from='client.24304 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi047=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:00:48.447 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:48 smithi047 ceph-mon[39856]: Saving service rgw.foo.a spec with placement smithi047=foo.a;count:1 2024-09-10T19:00:48.447 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:48 smithi047 ceph-mon[39856]: Deploying daemon rgw.foo.a on smithi047 2024-09-10T19:00:48.447 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:48 smithi047 ceph-mon[29019]: from='client.24304 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi047=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:00:48.447 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:48 smithi047 ceph-mon[29019]: Saving service rgw.foo.a spec with placement smithi047=foo.a;count:1 2024-09-10T19:00:48.447 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:48 smithi047 ceph-mon[29019]: Deploying daemon rgw.foo.a on smithi047 2024-09-10T19:00:48.789 INFO:journalctl@ceph.rgw.foo.a.smithi047.stdout:Sep 10 19:00:48 smithi047 systemd[1]: Starting Ceph rgw.foo.a for d8563d7e-6fa5-11ef-bcea-c7b262605968... 2024-09-10T19:00:49.222 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:49 smithi047 ceph-mon[29019]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:00:49.223 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:49 smithi047 ceph-mon[39856]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:00:49.223 INFO:journalctl@ceph.rgw.foo.a.smithi047.stdout:Sep 10 19:00:48 smithi047 podman[65536]: 2024-09-10 19:00:48.886456058 +0000 UTC m=+0.110270866 container create ae5fc6a28e182e68eabc39234247f98905adb5a7460d6b769481def95c00b3b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-rgw-foo-a, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T19:00:49.223 INFO:journalctl@ceph.rgw.foo.a.smithi047.stdout:Sep 10 19:00:48 smithi047 podman[65536]: 2024-09-10 19:00:48.792467927 +0000 UTC m=+0.016282747 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16 2024-09-10T19:00:49.223 INFO:journalctl@ceph.rgw.foo.a.smithi047.stdout:Sep 10 19:00:49 smithi047 podman[65536]: 2024-09-10 19:00:49.122307017 +0000 UTC m=+0.346121825 container init ae5fc6a28e182e68eabc39234247f98905adb5a7460d6b769481def95c00b3b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-rgw-foo-a, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739) 2024-09-10T19:00:49.223 INFO:journalctl@ceph.rgw.foo.a.smithi047.stdout:Sep 10 19:00:49 smithi047 podman[65536]: 2024-09-10 19:00:49.125447416 +0000 UTC m=+0.349262235 container start ae5fc6a28e182e68eabc39234247f98905adb5a7460d6b769481def95c00b3b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-rgw-foo-a, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-10T19:00:49.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:49 smithi149 ceph-mon[33230]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:00:49.539 INFO:journalctl@ceph.rgw.foo.a.smithi047.stdout:Sep 10 19:00:49 smithi047 bash[65536]: ae5fc6a28e182e68eabc39234247f98905adb5a7460d6b769481def95c00b3b7 2024-09-10T19:00:49.539 INFO:journalctl@ceph.rgw.foo.a.smithi047.stdout:Sep 10 19:00:49 smithi047 systemd[1]: Started Ceph rgw.foo.a for d8563d7e-6fa5-11ef-bcea-c7b262605968. 2024-09-10T19:00:50.053 INFO:teuthology.orchestra.run.smithi149.stderr:pool 'datapool' created 2024-09-10T19:00:50.342 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:50 smithi149 ceph-mon[33230]: from='client.? 172.21.15.149:0/2646448423' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-10T19:00:50.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:50 smithi149 ceph-mon[33230]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-10T19:00:50.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:50 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:50.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:50 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:50.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:50 smithi047 ceph-mon[29019]: from='client.? 172.21.15.149:0/2646448423' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-10T19:00:50.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:50 smithi047 ceph-mon[29019]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-10T19:00:50.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:50 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:50.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:50 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:50.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:50 smithi047 ceph-mon[39856]: from='client.? 172.21.15.149:0/2646448423' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-10T19:00:50.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:50 smithi047 ceph-mon[39856]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-10T19:00:50.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:50 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:50.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:50 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:50.587 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8563d7e-6fa5-11ef-bcea-c7b262605968 -- rbd pool init datapool 2024-09-10T19:00:50.831 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T19:00:51.274 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:51 smithi149 ceph-mon[33230]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:00:51.274 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:51 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:51.274 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:51 smithi149 ceph-mon[33230]: Saving service rgw.foo.a spec with placement smithi047=foo.a;count:1 2024-09-10T19:00:51.274 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:51 smithi149 ceph-mon[33230]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-10T19:00:51.274 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:51 smithi149 ceph-mon[33230]: osdmap e52: 8 total, 8 up, 8 in 2024-09-10T19:00:51.275 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:51 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:51.275 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:51 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/2288122300' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-10T19:00:51.275 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:51 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:51.275 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:51 smithi149 ceph-mon[33230]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-10T19:00:51.275 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:51 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:00:51.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:51 smithi047 ceph-mon[29019]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:00:51.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:51 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:51.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:51 smithi047 ceph-mon[29019]: Saving service rgw.foo.a spec with placement smithi047=foo.a;count:1 2024-09-10T19:00:51.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:51 smithi047 ceph-mon[29019]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-10T19:00:51.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:51 smithi047 ceph-mon[29019]: osdmap e52: 8 total, 8 up, 8 in 2024-09-10T19:00:51.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:51 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:51.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:51 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/2288122300' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-10T19:00:51.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:51 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:51.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:51 smithi047 ceph-mon[29019]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-10T19:00:51.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:51 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:00:51.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:51 smithi047 ceph-mon[39856]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:00:51.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:51 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:51.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:51 smithi047 ceph-mon[39856]: Saving service rgw.foo.a spec with placement smithi047=foo.a;count:1 2024-09-10T19:00:51.541 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:51 smithi047 ceph-mon[39856]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-10T19:00:51.541 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:51 smithi047 ceph-mon[39856]: osdmap e52: 8 total, 8 up, 8 in 2024-09-10T19:00:51.541 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:51 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:51.541 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:51 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/2288122300' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-10T19:00:51.541 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:51 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:51.541 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:51 smithi047 ceph-mon[39856]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-10T19:00:51.541 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:51 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:00:52.214 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:52 smithi149 ceph-mon[33230]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-10T19:00:52.214 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:52 smithi149 ceph-mon[33230]: osdmap e53: 8 total, 8 up, 8 in 2024-09-10T19:00:52.214 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:52 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:52.214 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:52 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:52.215 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:52 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:00:52.215 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:52 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:00:52.539 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:52 smithi047 ceph-mon[39856]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-10T19:00:52.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:52 smithi047 ceph-mon[39856]: osdmap e53: 8 total, 8 up, 8 in 2024-09-10T19:00:52.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:52 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:52.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:52 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:52.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:52 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:00:52.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:52 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:00:52.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:52 smithi047 ceph-mon[29019]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-10T19:00:52.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:52 smithi047 ceph-mon[29019]: osdmap e53: 8 total, 8 up, 8 in 2024-09-10T19:00:52.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:52 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:52.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:52 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:52.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:52 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:00:52.541 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:52 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:00:53.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:53 smithi149 ceph-mon[33230]: Checking dashboard <-> RGW credentials 2024-09-10T19:00:53.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:53 smithi149 ceph-mon[33230]: pgmap v170: 36 pgs: 9 creating+peering, 23 unknown, 4 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:00:53.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:53 smithi149 ceph-mon[33230]: osdmap e54: 8 total, 8 up, 8 in 2024-09-10T19:00:53.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:53 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/2288122300' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-10T19:00:53.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:53 smithi149 ceph-mon[33230]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-10T19:00:53.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:53 smithi149 ceph-mon[33230]: from='client.? 172.21.15.149:0/632043185' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-10T19:00:53.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:53 smithi149 ceph-mon[33230]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-10T19:00:53.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:53 smithi149 ceph-mon[33230]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:00:53.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:53 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:53.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:53 smithi047 ceph-mon[29019]: Checking dashboard <-> RGW credentials 2024-09-10T19:00:53.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:53 smithi047 ceph-mon[29019]: pgmap v170: 36 pgs: 9 creating+peering, 23 unknown, 4 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:00:53.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:53 smithi047 ceph-mon[29019]: osdmap e54: 8 total, 8 up, 8 in 2024-09-10T19:00:53.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:53 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/2288122300' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-10T19:00:53.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:53 smithi047 ceph-mon[29019]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-10T19:00:53.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:53 smithi047 ceph-mon[29019]: from='client.? 172.21.15.149:0/632043185' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-10T19:00:53.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:53 smithi047 ceph-mon[29019]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-10T19:00:53.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:53 smithi047 ceph-mon[29019]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:00:53.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:53 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:53.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:53 smithi047 ceph-mon[39856]: Checking dashboard <-> RGW credentials 2024-09-10T19:00:53.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:53 smithi047 ceph-mon[39856]: pgmap v170: 36 pgs: 9 creating+peering, 23 unknown, 4 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:00:53.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:53 smithi047 ceph-mon[39856]: osdmap e54: 8 total, 8 up, 8 in 2024-09-10T19:00:53.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:53 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/2288122300' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-10T19:00:53.541 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:53 smithi047 ceph-mon[39856]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-10T19:00:53.541 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:53 smithi047 ceph-mon[39856]: from='client.? 172.21.15.149:0/632043185' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-10T19:00:53.541 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:53 smithi047 ceph-mon[39856]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-10T19:00:53.541 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:53 smithi047 ceph-mon[39856]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:00:53.541 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:53 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:54.342 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:54 smithi149 ceph-mon[33230]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-10T19:00:54.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:54 smithi149 ceph-mon[33230]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-10T19:00:54.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:54 smithi149 ceph-mon[33230]: osdmap e55: 8 total, 8 up, 8 in 2024-09-10T19:00:54.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:54 smithi047 ceph-mon[29019]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-10T19:00:54.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:54 smithi047 ceph-mon[29019]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-10T19:00:54.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:54 smithi047 ceph-mon[29019]: osdmap e55: 8 total, 8 up, 8 in 2024-09-10T19:00:54.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:54 smithi047 ceph-mon[39856]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-10T19:00:54.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:54 smithi047 ceph-mon[39856]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-10T19:00:54.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:54 smithi047 ceph-mon[39856]: osdmap e55: 8 total, 8 up, 8 in 2024-09-10T19:00:55.342 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:55 smithi149 ceph-mon[33230]: pgmap v173: 68 pgs: 2 creating+activating, 10 creating+peering, 36 unknown, 20 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 767 B/s wr, 3 op/s 2024-09-10T19:00:55.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:55 smithi149 ceph-mon[33230]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-10T19:00:55.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:55 smithi149 ceph-mon[33230]: Cluster is now healthy 2024-09-10T19:00:55.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:55 smithi149 ceph-mon[33230]: osdmap e56: 8 total, 8 up, 8 in 2024-09-10T19:00:55.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:55 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/2288122300' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-10T19:00:55.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:55 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/1939605258' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-10T19:00:55.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:55 smithi149 ceph-mon[33230]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-10T19:00:55.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:55 smithi149 ceph-mon[33230]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-10T19:00:55.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:55 smithi047 ceph-mon[29019]: pgmap v173: 68 pgs: 2 creating+activating, 10 creating+peering, 36 unknown, 20 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 767 B/s wr, 3 op/s 2024-09-10T19:00:55.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:55 smithi047 ceph-mon[29019]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-10T19:00:55.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:55 smithi047 ceph-mon[29019]: Cluster is now healthy 2024-09-10T19:00:55.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:55 smithi047 ceph-mon[29019]: osdmap e56: 8 total, 8 up, 8 in 2024-09-10T19:00:55.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:55 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/2288122300' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-10T19:00:55.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:55 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/1939605258' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-10T19:00:55.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:55 smithi047 ceph-mon[29019]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-10T19:00:55.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:55 smithi047 ceph-mon[29019]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-10T19:00:55.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:55 smithi047 ceph-mon[39856]: pgmap v173: 68 pgs: 2 creating+activating, 10 creating+peering, 36 unknown, 20 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 767 B/s wr, 3 op/s 2024-09-10T19:00:55.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:55 smithi047 ceph-mon[39856]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-10T19:00:55.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:55 smithi047 ceph-mon[39856]: Cluster is now healthy 2024-09-10T19:00:55.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:55 smithi047 ceph-mon[39856]: osdmap e56: 8 total, 8 up, 8 in 2024-09-10T19:00:55.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:55 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/2288122300' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-10T19:00:55.541 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:55 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/1939605258' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-10T19:00:55.541 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:55 smithi047 ceph-mon[39856]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-10T19:00:55.541 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:55 smithi047 ceph-mon[39856]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-10T19:00:55.701 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8563d7e-6fa5-11ef-bcea-c7b262605968 -- ceph orch apply iscsi datapool admin admin --trusted_ip_list 172.21.15.149 --placement '1;smithi149=iscsi.a' 2024-09-10T19:00:55.942 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T19:00:56.337 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:56 smithi149 ceph-mon[33230]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-10T19:00:56.337 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:56 smithi149 ceph-mon[33230]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-10T19:00:56.337 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:56 smithi149 ceph-mon[33230]: osdmap e57: 8 total, 8 up, 8 in 2024-09-10T19:00:56.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:56 smithi047 ceph-mon[29019]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-10T19:00:56.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:56 smithi047 ceph-mon[29019]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-10T19:00:56.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:56 smithi047 ceph-mon[29019]: osdmap e57: 8 total, 8 up, 8 in 2024-09-10T19:00:56.539 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:56 smithi047 ceph-mon[39856]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-10T19:00:56.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:56 smithi047 ceph-mon[39856]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-10T19:00:56.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:56 smithi047 ceph-mon[39856]: osdmap e57: 8 total, 8 up, 8 in 2024-09-10T19:00:57.319 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:57 smithi149 ceph-mon[33230]: pgmap v176: 100 pgs: 2 creating+activating, 5 creating+peering, 35 unknown, 58 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 4.5 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-09-10T19:00:57.319 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:57 smithi149 ceph-mon[33230]: osdmap e58: 8 total, 8 up, 8 in 2024-09-10T19:00:57.320 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:57 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/2942656226' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-10T19:00:57.320 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:57 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/4189189335' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-10T19:00:57.320 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:57 smithi149 ceph-mon[33230]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-10T19:00:57.539 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:57 smithi047 ceph-mon[39856]: pgmap v176: 100 pgs: 2 creating+activating, 5 creating+peering, 35 unknown, 58 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 4.5 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-09-10T19:00:57.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:57 smithi047 ceph-mon[39856]: osdmap e58: 8 total, 8 up, 8 in 2024-09-10T19:00:57.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:57 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/2942656226' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-10T19:00:57.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:57 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/4189189335' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-10T19:00:57.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:57 smithi047 ceph-mon[39856]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-10T19:00:57.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:57 smithi047 ceph-mon[29019]: pgmap v176: 100 pgs: 2 creating+activating, 5 creating+peering, 35 unknown, 58 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 4.5 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-09-10T19:00:57.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:57 smithi047 ceph-mon[29019]: osdmap e58: 8 total, 8 up, 8 in 2024-09-10T19:00:57.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:57 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/2942656226' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-10T19:00:57.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:57 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/4189189335' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-10T19:00:57.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:57 smithi047 ceph-mon[29019]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-10T19:00:57.701 INFO:teuthology.orchestra.run.smithi149.stdout:Scheduled iscsi.datapool update... 2024-09-10T19:00:58.342 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:58 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/2942656226' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-10T19:00:58.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:58 smithi149 ceph-mon[33230]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-10T19:00:58.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:58 smithi149 ceph-mon[33230]: osdmap e59: 8 total, 8 up, 8 in 2024-09-10T19:00:58.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:58 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/2942656226' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-10T19:00:58.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:58 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/4189189335' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-10T19:00:58.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:58 smithi149 ceph-mon[33230]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-10T19:00:58.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:58 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:58.467 INFO:tasks.cephadm:Distributing iscsi-gateway.cfg... 2024-09-10T19:00:58.468 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-09-10T19:00:58.468 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2024-09-10T19:00:58.491 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:58 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/2942656226' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-10T19:00:58.492 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:58 smithi047 ceph-mon[39856]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-10T19:00:58.492 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:58 smithi047 ceph-mon[39856]: osdmap e59: 8 total, 8 up, 8 in 2024-09-10T19:00:58.492 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:58 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/2942656226' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-10T19:00:58.492 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:58 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/4189189335' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-10T19:00:58.492 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:58 smithi047 ceph-mon[39856]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-10T19:00:58.492 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:58 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:58.492 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:58 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/2942656226' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-10T19:00:58.492 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:58 smithi047 ceph-mon[29019]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-10T19:00:58.492 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:58 smithi047 ceph-mon[29019]: osdmap e59: 8 total, 8 up, 8 in 2024-09-10T19:00:58.492 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:58 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/2942656226' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-10T19:00:58.492 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:58 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/4189189335' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-10T19:00:58.493 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:58 smithi047 ceph-mon[29019]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-10T19:00:58.493 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:58 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:58.496 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-09-10T19:00:58.497 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2024-09-10T19:00:58.525 DEBUG:teuthology.orchestra.run.smithi149:iscsi.iscsi.a> sudo journalctl -f -n 0 -u ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968@iscsi.iscsi.a.service 2024-09-10T19:00:58.568 INFO:tasks.cephadm:Adding prometheus.a on smithi149 2024-09-10T19:00:58.569 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8563d7e-6fa5-11ef-bcea-c7b262605968 -- ceph orch apply prometheus '1;smithi149=a' 2024-09-10T19:00:59.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:59 smithi149 ceph-mon[33230]: from='client.24361 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.149", "placement": "1;smithi149=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:00:59.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:59 smithi149 ceph-mon[33230]: Saving service iscsi.datapool spec with placement smithi149=iscsi.a;count:1 2024-09-10T19:00:59.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:59 smithi149 ceph-mon[33230]: pgmap v179: 132 pgs: 2 creating+activating, 14 creating+peering, 29 unknown, 87 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.0 KiB/s rd, 1023 B/s wr, 4 op/s 2024-09-10T19:00:59.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:59 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/2942656226' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-10T19:00:59.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:59 smithi149 ceph-mon[33230]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-10T19:00:59.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:59 smithi149 ceph-mon[33230]: osdmap e60: 8 total, 8 up, 8 in 2024-09-10T19:00:59.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:59 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:59.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:59 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:59.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:59 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:59.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:59 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-09-10T19:00:59.344 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:59 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-10T19:00:59.344 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:59 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:00:59.344 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:00:59 smithi149 ceph-mon[33230]: Deploying daemon iscsi.iscsi.a on smithi149 2024-09-10T19:00:59.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:59 smithi047 ceph-mon[29019]: from='client.24361 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.149", "placement": "1;smithi149=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:00:59.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:59 smithi047 ceph-mon[29019]: Saving service iscsi.datapool spec with placement smithi149=iscsi.a;count:1 2024-09-10T19:00:59.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:59 smithi047 ceph-mon[29019]: pgmap v179: 132 pgs: 2 creating+activating, 14 creating+peering, 29 unknown, 87 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.0 KiB/s rd, 1023 B/s wr, 4 op/s 2024-09-10T19:00:59.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:59 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/2942656226' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-10T19:00:59.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:59 smithi047 ceph-mon[29019]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-10T19:00:59.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:59 smithi047 ceph-mon[29019]: osdmap e60: 8 total, 8 up, 8 in 2024-09-10T19:00:59.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:59 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:59.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:59 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:59.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:59 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:59.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:59 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-09-10T19:00:59.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:59 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-10T19:00:59.541 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:59 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:00:59.541 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:00:59 smithi047 ceph-mon[29019]: Deploying daemon iscsi.iscsi.a on smithi149 2024-09-10T19:00:59.541 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:59 smithi047 ceph-mon[39856]: from='client.24361 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.149", "placement": "1;smithi149=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:00:59.541 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:59 smithi047 ceph-mon[39856]: Saving service iscsi.datapool spec with placement smithi149=iscsi.a;count:1 2024-09-10T19:00:59.541 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:59 smithi047 ceph-mon[39856]: pgmap v179: 132 pgs: 2 creating+activating, 14 creating+peering, 29 unknown, 87 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.0 KiB/s rd, 1023 B/s wr, 4 op/s 2024-09-10T19:00:59.541 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:59 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/2942656226' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-10T19:00:59.541 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:59 smithi047 ceph-mon[39856]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-10T19:00:59.541 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:59 smithi047 ceph-mon[39856]: osdmap e60: 8 total, 8 up, 8 in 2024-09-10T19:00:59.541 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:59 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:59.541 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:59 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:59.541 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:59 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:00:59.542 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:59 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-09-10T19:00:59.542 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:59 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-10T19:00:59.542 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:59 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:00:59.542 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:00:59 smithi047 ceph-mon[39856]: Deploying daemon iscsi.iscsi.a on smithi149 2024-09-10T19:00:59.677 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T19:01:01.327 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:01 smithi149 ceph-mon[33230]: pgmap v181: 132 pgs: 10 creating+peering, 7 unknown, 115 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 26 KiB/s rd, 3.1 KiB/s wr, 60 op/s 2024-09-10T19:01:01.539 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:01 smithi047 ceph-mon[39856]: pgmap v181: 132 pgs: 10 creating+peering, 7 unknown, 115 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 26 KiB/s rd, 3.1 KiB/s wr, 60 op/s 2024-09-10T19:01:01.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:01 smithi047 ceph-mon[29019]: pgmap v181: 132 pgs: 10 creating+peering, 7 unknown, 115 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 26 KiB/s rd, 3.1 KiB/s wr, 60 op/s 2024-09-10T19:01:01.656 INFO:teuthology.orchestra.run.smithi149.stdout:Scheduled prometheus update... 2024-09-10T19:01:02.401 DEBUG:teuthology.orchestra.run.smithi149:prometheus.a> sudo journalctl -f -n 0 -u ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968@prometheus.a.service 2024-09-10T19:01:02.403 INFO:tasks.cephadm:Adding node-exporter.a on smithi047 2024-09-10T19:01:02.403 INFO:tasks.cephadm:Adding node-exporter.b on smithi149 2024-09-10T19:01:02.404 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8563d7e-6fa5-11ef-bcea-c7b262605968 -- ceph orch apply node-exporter '2;smithi047=a;smithi149=b' 2024-09-10T19:01:02.660 INFO:journalctl@ceph.iscsi.iscsi.a.smithi149.stdout:Sep 10 19:01:02 smithi149 podman[60472]: 2024-09-10 19:01:02.590330166 +0000 UTC m=+0.560522919 container create 1167f4d9967915690ec2801e3a750df65b81609f79f10f7046fb5ddfd5e40185 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-iscsi-iscsi-a, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-10T19:01:02.957 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T19:01:03.039 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:02 smithi047 ceph-mon[29019]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi149=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:01:03.039 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:02 smithi047 ceph-mon[29019]: Saving service prometheus spec with placement smithi149=a;count:1 2024-09-10T19:01:03.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:02 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:01:03.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:02 smithi047 ceph-mon[29019]: pgmap v182: 132 pgs: 5 creating+peering, 127 active+clean; 581 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 54 KiB/s rd, 4.2 KiB/s wr, 131 op/s 2024-09-10T19:01:03.040 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:02 smithi047 ceph-mon[39856]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi149=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:01:03.040 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:02 smithi047 ceph-mon[39856]: Saving service prometheus spec with placement smithi149=a;count:1 2024-09-10T19:01:03.040 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:02 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:01:03.040 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:02 smithi047 ceph-mon[39856]: pgmap v182: 132 pgs: 5 creating+peering, 127 active+clean; 581 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 54 KiB/s rd, 4.2 KiB/s wr, 131 op/s 2024-09-10T19:01:03.078 INFO:journalctl@ceph.iscsi.iscsi.a.smithi149.stdout:Sep 10 19:01:02 smithi149 podman[60472]: 2024-09-10 19:01:02.817510941 +0000 UTC m=+0.787703693 container init 1167f4d9967915690ec2801e3a750df65b81609f79f10f7046fb5ddfd5e40185 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-iscsi-iscsi-a, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-10T19:01:03.078 INFO:journalctl@ceph.iscsi.iscsi.a.smithi149.stdout:Sep 10 19:01:02 smithi149 podman[60472]: 2024-09-10 19:01:02.821028545 +0000 UTC m=+0.791221298 container start 1167f4d9967915690ec2801e3a750df65b81609f79f10f7046fb5ddfd5e40185 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-iscsi-iscsi-a, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-10T19:01:03.078 INFO:journalctl@ceph.iscsi.iscsi.a.smithi149.stdout:Sep 10 19:01:02 smithi149 bash[60472]: 1167f4d9967915690ec2801e3a750df65b81609f79f10f7046fb5ddfd5e40185 2024-09-10T19:01:03.078 INFO:journalctl@ceph.iscsi.iscsi.a.smithi149.stdout:Sep 10 19:01:02 smithi149 systemd[1]: Started Ceph iscsi.iscsi.a for d8563d7e-6fa5-11ef-bcea-c7b262605968. 2024-09-10T19:01:03.078 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:02 smithi149 ceph-mon[33230]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi149=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:01:03.078 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:02 smithi149 ceph-mon[33230]: Saving service prometheus spec with placement smithi149=a;count:1 2024-09-10T19:01:03.078 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:02 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:01:03.078 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:02 smithi149 ceph-mon[33230]: pgmap v182: 132 pgs: 5 creating+peering, 127 active+clean; 581 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 54 KiB/s rd, 4.2 KiB/s wr, 131 op/s 2024-09-10T19:01:03.330 INFO:journalctl@ceph.iscsi.iscsi.a.smithi149.stdout:Sep 10 19:01:03 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-iscsi-iscsi-a[60538]: debug Started the configuration object watcher 2024-09-10T19:01:03.330 INFO:journalctl@ceph.iscsi.iscsi.a.smithi149.stdout:Sep 10 19:01:03 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-iscsi-iscsi-a[60538]: debug Checking for config object changes every 1s 2024-09-10T19:01:03.330 INFO:journalctl@ceph.iscsi.iscsi.a.smithi149.stdout:Sep 10 19:01:03 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-iscsi-iscsi-a[60538]: debug Processing osd blocklist entries for this node 2024-09-10T19:01:03.593 INFO:journalctl@ceph.iscsi.iscsi.a.smithi149.stdout:Sep 10 19:01:03 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-iscsi-iscsi-a[60538]: debug Reading the configuration object to update local LIO configuration 2024-09-10T19:01:03.593 INFO:journalctl@ceph.iscsi.iscsi.a.smithi149.stdout:Sep 10 19:01:03 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-iscsi-iscsi-a[60538]: debug Configuration does not have an entry for this host(smithi149.front.sepia.ceph.com) - nothing to define to LIO 2024-09-10T19:01:03.593 INFO:journalctl@ceph.iscsi.iscsi.a.smithi149.stdout:Sep 10 19:01:03 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-iscsi-iscsi-a[60538]: * Serving Flask app 'rbd-target-api' (lazy loading) 2024-09-10T19:01:03.593 INFO:journalctl@ceph.iscsi.iscsi.a.smithi149.stdout:Sep 10 19:01:03 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-iscsi-iscsi-a[60538]: * Environment: production 2024-09-10T19:01:03.593 INFO:journalctl@ceph.iscsi.iscsi.a.smithi149.stdout:Sep 10 19:01:03 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-iscsi-iscsi-a[60538]: WARNING: This is a development server. Do not use it in a production deployment. 2024-09-10T19:01:03.593 INFO:journalctl@ceph.iscsi.iscsi.a.smithi149.stdout:Sep 10 19:01:03 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-iscsi-iscsi-a[60538]: Use a production WSGI server instead. 2024-09-10T19:01:03.594 INFO:journalctl@ceph.iscsi.iscsi.a.smithi149.stdout:Sep 10 19:01:03 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-iscsi-iscsi-a[60538]: * Debug mode: off 2024-09-10T19:01:03.594 INFO:journalctl@ceph.iscsi.iscsi.a.smithi149.stdout:Sep 10 19:01:03 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-iscsi-iscsi-a[60538]: debug * Running on all addresses. 2024-09-10T19:01:03.594 INFO:journalctl@ceph.iscsi.iscsi.a.smithi149.stdout:Sep 10 19:01:03 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-iscsi-iscsi-a[60538]: WARNING: This is a development server. Do not use it in a production deployment. 2024-09-10T19:01:03.594 INFO:journalctl@ceph.iscsi.iscsi.a.smithi149.stdout:Sep 10 19:01:03 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-iscsi-iscsi-a[60538]: * Running on all addresses. 2024-09-10T19:01:03.594 INFO:journalctl@ceph.iscsi.iscsi.a.smithi149.stdout:Sep 10 19:01:03 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-iscsi-iscsi-a[60538]: WARNING: This is a development server. Do not use it in a production deployment. 2024-09-10T19:01:03.594 INFO:journalctl@ceph.iscsi.iscsi.a.smithi149.stdout:Sep 10 19:01:03 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-iscsi-iscsi-a[60538]: debug * Running on http://[fe80::ec4:7aff:fe88:7d57]:5000/ (Press CTRL+C to quit) 2024-09-10T19:01:03.594 INFO:journalctl@ceph.iscsi.iscsi.a.smithi149.stdout:Sep 10 19:01:03 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-iscsi-iscsi-a[60538]: * Running on http://[fe80::ec4:7aff:fe88:7d57]:5000/ (Press CTRL+C to quit) 2024-09-10T19:01:04.092 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:03 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:01:04.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:03 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:01:04.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:03 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:01:04.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:03 smithi149 ceph-mon[33230]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-10T19:01:04.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:03 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:01:04.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:03 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:01:04.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:03 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:01:04.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:03 smithi149 ceph-mon[33230]: from='client.? 172.21.15.149:0/1758111037' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T19:01:04.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:03 smithi149 ceph-mon[33230]: osdmap e61: 8 total, 8 up, 8 in 2024-09-10T19:01:04.289 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:03 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:01:04.289 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:03 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:01:04.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:03 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:01:04.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:03 smithi047 ceph-mon[29019]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-10T19:01:04.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:03 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:01:04.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:03 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:01:04.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:03 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:01:04.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:03 smithi047 ceph-mon[29019]: from='client.? 172.21.15.149:0/1758111037' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T19:01:04.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:03 smithi047 ceph-mon[29019]: osdmap e61: 8 total, 8 up, 8 in 2024-09-10T19:01:04.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:03 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:01:04.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:03 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:01:04.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:03 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:01:04.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:03 smithi047 ceph-mon[39856]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-10T19:01:04.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:03 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:01:04.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:03 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:01:04.291 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:03 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:01:04.291 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:03 smithi047 ceph-mon[39856]: from='client.? 172.21.15.149:0/1758111037' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T19:01:04.291 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:03 smithi047 ceph-mon[39856]: osdmap e61: 8 total, 8 up, 8 in 2024-09-10T19:01:04.686 INFO:teuthology.orchestra.run.smithi149.stdout:Scheduled node-exporter update... 2024-09-10T19:01:05.289 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:04 smithi047 ceph-mon[29019]: pgmap v184: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 67 KiB/s rd, 5.1 KiB/s wr, 162 op/s 2024-09-10T19:01:05.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:04 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:01:05.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:04 smithi047 ceph-mon[39856]: pgmap v184: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 67 KiB/s rd, 5.1 KiB/s wr, 162 op/s 2024-09-10T19:01:05.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:04 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:01:05.327 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:04 smithi149 ceph-mon[33230]: pgmap v184: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 67 KiB/s rd, 5.1 KiB/s wr, 162 op/s 2024-09-10T19:01:05.327 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:04 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:01:05.876 DEBUG:teuthology.orchestra.run.smithi047:node-exporter.a> sudo journalctl -f -n 0 -u ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968@node-exporter.a.service 2024-09-10T19:01:05.879 DEBUG:teuthology.orchestra.run.smithi149:node-exporter.b> sudo journalctl -f -n 0 -u ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968@node-exporter.b.service 2024-09-10T19:01:05.881 INFO:tasks.cephadm:Adding alertmanager.a on smithi047 2024-09-10T19:01:05.881 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8563d7e-6fa5-11ef-bcea-c7b262605968 -- ceph orch apply alertmanager '1;smithi047=a' 2024-09-10T19:01:06.273 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:05 smithi149 ceph-mon[33230]: from='client.24415 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi047=a;smithi149=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:01:06.311 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:05 smithi149 ceph-mon[33230]: Saving service node-exporter spec with placement smithi047=a;smithi149=b;count:2 2024-09-10T19:01:06.311 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:05 smithi149 ceph-mon[33230]: mgrmap e16: y(active, since 4m), standbys: x 2024-09-10T19:01:06.311 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:05 smithi047 ceph-mon[29019]: from='client.24415 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi047=a;smithi149=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:01:06.311 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:05 smithi047 ceph-mon[29019]: Saving service node-exporter spec with placement smithi047=a;smithi149=b;count:2 2024-09-10T19:01:06.311 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:05 smithi047 ceph-mon[29019]: mgrmap e16: y(active, since 4m), standbys: x 2024-09-10T19:01:06.312 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:05 smithi047 ceph-mon[39856]: from='client.24415 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi047=a;smithi149=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:01:06.312 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:05 smithi047 ceph-mon[39856]: Saving service node-exporter spec with placement smithi047=a;smithi149=b;count:2 2024-09-10T19:01:06.312 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:05 smithi047 ceph-mon[39856]: mgrmap e16: y(active, since 4m), standbys: x 2024-09-10T19:01:06.836 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T19:01:07.234 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:06 smithi149 ceph-mon[33230]: pgmap v185: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 58 KiB/s rd, 4.5 KiB/s wr, 141 op/s 2024-09-10T19:01:07.234 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:06 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:01:07.234 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:06 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:01:07.234 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:06 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:01:07.234 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:06 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:01:07.234 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:06 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:01:07.289 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:06 smithi047 ceph-mon[29019]: pgmap v185: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 58 KiB/s rd, 4.5 KiB/s wr, 141 op/s 2024-09-10T19:01:07.289 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:06 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:01:07.289 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:06 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:01:07.289 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:06 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:01:07.289 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:06 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:01:07.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:06 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:01:07.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:06 smithi047 ceph-mon[39856]: pgmap v185: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 58 KiB/s rd, 4.5 KiB/s wr, 141 op/s 2024-09-10T19:01:07.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:06 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:01:07.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:06 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:01:07.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:06 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:01:07.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:06 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:01:07.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:06 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:01:08.135 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:07 smithi149 ceph-mon[33230]: Deploying daemon prometheus.a on smithi149 2024-09-10T19:01:08.289 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:07 smithi047 ceph-mon[29019]: Deploying daemon prometheus.a on smithi149 2024-09-10T19:01:08.289 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:07 smithi047 ceph-mon[39856]: Deploying daemon prometheus.a on smithi149 2024-09-10T19:01:08.533 INFO:teuthology.orchestra.run.smithi149.stdout:Scheduled alertmanager update... 2024-09-10T19:01:09.289 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:08 smithi047 ceph-mon[29019]: pgmap v186: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 47 KiB/s rd, 3.6 KiB/s wr, 114 op/s 2024-09-10T19:01:09.289 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:08 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:01:09.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:08 smithi047 ceph-mon[39856]: pgmap v186: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 47 KiB/s rd, 3.6 KiB/s wr, 114 op/s 2024-09-10T19:01:09.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:08 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:01:09.342 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:08 smithi149 ceph-mon[33230]: pgmap v186: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 47 KiB/s rd, 3.6 KiB/s wr, 114 op/s 2024-09-10T19:01:09.342 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:08 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:01:09.453 DEBUG:teuthology.orchestra.run.smithi047:alertmanager.a> sudo journalctl -f -n 0 -u ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968@alertmanager.a.service 2024-09-10T19:01:09.456 INFO:tasks.cephadm:Adding grafana.a on smithi149 2024-09-10T19:01:09.456 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8563d7e-6fa5-11ef-bcea-c7b262605968 -- ceph orch apply grafana '1;smithi149=a' 2024-09-10T19:01:09.732 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T19:01:10.231 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:09 smithi149 ceph-mon[33230]: from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi047=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:01:10.231 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:09 smithi149 ceph-mon[33230]: Saving service alertmanager spec with placement smithi047=a;count:1 2024-09-10T19:01:10.289 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:09 smithi047 ceph-mon[29019]: from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi047=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:01:10.289 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:09 smithi047 ceph-mon[29019]: Saving service alertmanager spec with placement smithi047=a;count:1 2024-09-10T19:01:10.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:09 smithi047 ceph-mon[39856]: from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi047=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:01:10.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:09 smithi047 ceph-mon[39856]: Saving service alertmanager spec with placement smithi047=a;count:1 2024-09-10T19:01:11.289 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:10 smithi047 ceph-mon[29019]: pgmap v187: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 35 KiB/s rd, 2.5 KiB/s wr, 85 op/s 2024-09-10T19:01:11.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:10 smithi047 ceph-mon[39856]: pgmap v187: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 35 KiB/s rd, 2.5 KiB/s wr, 85 op/s 2024-09-10T19:01:11.298 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:10 smithi149 ceph-mon[33230]: pgmap v187: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 35 KiB/s rd, 2.5 KiB/s wr, 85 op/s 2024-09-10T19:01:12.079 INFO:teuthology.orchestra.run.smithi149.stdout:Scheduled grafana update... 2024-09-10T19:01:12.957 DEBUG:teuthology.orchestra.run.smithi149:grafana.a> sudo journalctl -f -n 0 -u ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968@grafana.a.service 2024-09-10T19:01:12.959 INFO:tasks.cephadm:Setting up client nodes... 2024-09-10T19:01:12.960 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8563d7e-6fa5-11ef-bcea-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-10T19:01:13.230 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:01:13.289 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:13 smithi047 ceph-mon[29019]: pgmap v188: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 15 KiB/s rd, 1.1 KiB/s wr, 35 op/s 2024-09-10T19:01:13.289 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:13 smithi047 ceph-mon[29019]: from='client.24427 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi149=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:01:13.289 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:13 smithi047 ceph-mon[29019]: Saving service grafana spec with placement smithi149=a;count:1 2024-09-10T19:01:13.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:13 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:01:13.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:13 smithi047 ceph-mon[39856]: pgmap v188: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 15 KiB/s rd, 1.1 KiB/s wr, 35 op/s 2024-09-10T19:01:13.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:13 smithi047 ceph-mon[39856]: from='client.24427 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi149=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:01:13.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:13 smithi047 ceph-mon[39856]: Saving service grafana spec with placement smithi149=a;count:1 2024-09-10T19:01:13.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:13 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:01:13.343 INFO:journalctl@ceph.iscsi.iscsi.a.smithi149.stdout:Sep 10 19:01:13 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-iscsi-iscsi-a[60538]: debug there is no tcmu-runner data available 2024-09-10T19:01:13.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:13 smithi149 ceph-mon[33230]: pgmap v188: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 15 KiB/s rd, 1.1 KiB/s wr, 35 op/s 2024-09-10T19:01:13.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:13 smithi149 ceph-mon[33230]: from='client.24427 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi149=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:01:13.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:13 smithi149 ceph-mon[33230]: Saving service grafana spec with placement smithi149=a;count:1 2024-09-10T19:01:13.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:13 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:01:14.342 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:14 smithi149 ceph-mon[33230]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:01:14.363 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:14 smithi047 ceph-mon[29019]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:01:14.364 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:14 smithi047 ceph-mon[39856]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:01:15.108 INFO:teuthology.orchestra.run.smithi047.stdout:[client.0] 2024-09-10T19:01:15.109 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQB7l+BmfgY6BhAAwAUAgkzAi0n8EogqDpP4Kg== 2024-09-10T19:01:15.342 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:15 smithi149 ceph-mon[33230]: pgmap v189: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 684 B/s rd, 97 B/s wr, 1 op/s 2024-09-10T19:01:15.422 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:15 smithi047 ceph-mon[29019]: pgmap v189: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 684 B/s rd, 97 B/s wr, 1 op/s 2024-09-10T19:01:15.423 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:15 smithi047 ceph-mon[39856]: pgmap v189: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 684 B/s rd, 97 B/s wr, 1 op/s 2024-09-10T19:01:15.759 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-09-10T19:01:15.759 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-10T19:01:15.759 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-10T19:01:15.801 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8563d7e-6fa5-11ef-bcea-c7b262605968 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-10T19:01:16.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:16 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/560245789' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T19:01:16.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:16 smithi149 ceph-mon[33230]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T19:01:16.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:16 smithi149 ceph-mon[33230]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T19:01:16.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:16 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/560245789' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T19:01:16.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:16 smithi047 ceph-mon[29019]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T19:01:16.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:16 smithi047 ceph-mon[29019]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T19:01:16.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:16 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/560245789' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T19:01:16.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:16 smithi047 ceph-mon[39856]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T19:01:16.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:16 smithi047 ceph-mon[39856]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T19:01:16.729 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T19:01:17.263 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:17 smithi149 ceph-mon[33230]: pgmap v190: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 85 B/s wr, 1 op/s 2024-09-10T19:01:17.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:17 smithi047 ceph-mon[29019]: pgmap v190: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 85 B/s wr, 1 op/s 2024-09-10T19:01:17.539 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:17 smithi047 ceph-mon[39856]: pgmap v190: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 85 B/s wr, 1 op/s 2024-09-10T19:01:19.283 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:19 smithi149 ceph-mon[33230]: pgmap v191: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:01:19.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:19 smithi047 ceph-mon[29019]: pgmap v191: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:01:19.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:19 smithi047 ceph-mon[39856]: pgmap v191: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:01:20.040 INFO:teuthology.orchestra.run.smithi149.stdout:[client.1] 2024-09-10T19:01:20.040 INFO:teuthology.orchestra.run.smithi149.stdout: key = AQCAl+BmQqANAhAA7e5rhafSt+KpDkedRJ3Q/g== 2024-09-10T19:01:20.313 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:20 smithi149 ceph-mon[33230]: from='client.? 172.21.15.149:0/2059107840' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T19:01:20.313 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:20 smithi149 ceph-mon[33230]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T19:01:20.313 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:20 smithi149 ceph-mon[33230]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T19:01:20.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:20 smithi047 ceph-mon[29019]: from='client.? 172.21.15.149:0/2059107840' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T19:01:20.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:20 smithi047 ceph-mon[29019]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T19:01:20.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:20 smithi047 ceph-mon[29019]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T19:01:20.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:20 smithi047 ceph-mon[39856]: from='client.? 172.21.15.149:0/2059107840' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T19:01:20.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:20 smithi047 ceph-mon[39856]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T19:01:20.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:20 smithi047 ceph-mon[39856]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T19:01:20.708 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-09-10T19:01:20.709 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-09-10T19:01:20.709 DEBUG:teuthology.orchestra.run.smithi149:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-10T19:01:20.749 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-10T19:01:20.750 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-10T19:01:20.750 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid d8563d7e-6fa5-11ef-bcea-c7b262605968 -- ceph mgr dump --format=json 2024-09-10T19:01:21.014 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:01:21.342 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:21 smithi149 ceph-mon[33230]: pgmap v192: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:01:21.343 INFO:journalctl@ceph.prometheus.a.smithi149.stdout:Sep 10 19:01:21 smithi149 systemd[1]: Started Ceph prometheus.a for d8563d7e-6fa5-11ef-bcea-c7b262605968. 2024-09-10T19:01:21.401 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:21 smithi047 ceph-mon[29019]: pgmap v192: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:01:21.401 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:21 smithi047 ceph-mon[39856]: pgmap v192: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:01:22.461 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:22 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:01:22.461 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:22 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:01:22.461 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:22 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:01:22.461 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:22 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-10T19:01:22.461 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:22 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:01:22.461 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:22 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:01:22.462 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:22 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:01:22.462 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:22 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-10T19:01:22.462 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:01:22 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: ignoring --setuser ceph since I am not root 2024-09-10T19:01:22.462 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:01:22 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: ignoring --setgroup ceph since I am not root 2024-09-10T19:01:22.462 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:01:22 smithi047 ceph-mgr[29248]: -- 172.21.15.47:0/2764480949 <== mon.2 v2:172.21.15.47:3301/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d08b3e8340 con 0x55d08b3f5000 2024-09-10T19:01:22.462 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:01:22 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:01:22.402+0000 7f569cb8aec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T19:01:22.462 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:01:22 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:01:22.459+0000 7f569cb8aec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T19:01:22.510 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:22 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:01:22.510 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:22 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:01:22.510 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:22 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' 2024-09-10T19:01:22.510 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:22 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-10T19:01:22.511 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:01:22 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: ignoring --setuser ceph since I am not root 2024-09-10T19:01:22.511 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:01:22 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: ignoring --setgroup ceph since I am not root 2024-09-10T19:01:22.511 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:01:22 smithi149 ceph-mgr[35094]: -- 172.21.15.149:0/1235435656 <== mon.1 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x564ef4866340 con 0x564ef4873000 2024-09-10T19:01:22.511 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:01:22 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:01:22.390+0000 7f00e223dec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T19:01:22.511 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:01:22 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:01:22.444+0000 7f00e223dec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T19:01:22.511 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:01:22 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:01:22.508+0000 7f00e223dec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T19:01:22.767 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:01:22 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:01:22.631+0000 7f00e223dec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T19:01:22.783 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:01:22 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:01:22.528+0000 7f569cb8aec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T19:01:22.783 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:01:22 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:01:22.659+0000 7f569cb8aec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T19:01:23.024 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:01:22 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:01:22.764+0000 7f00e223dec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T19:01:23.024 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:01:23 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:01:23.021+0000 7f00e223dec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T19:01:23.039 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:01:22 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:01:22.804+0000 7f569cb8aec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T19:01:23.095 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T19:01:23.342 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:01:23 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:01:23.084+0000 7f00e223dec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T19:01:23.343 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:01:23 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:01:23.245+0000 7f00e223dec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T19:01:23.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:23 smithi149 ceph-mon[33230]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-10T19:01:23.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:23 smithi149 ceph-mon[33230]: mgrmap e17: y(active, since 4m), standbys: x 2024-09-10T19:01:23.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:23 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/3889635798' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-10T19:01:23.411 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:23 smithi047 ceph-mon[29019]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-10T19:01:23.411 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:23 smithi047 ceph-mon[29019]: mgrmap e17: y(active, since 4m), standbys: x 2024-09-10T19:01:23.412 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:23 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/3889635798' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-10T19:01:23.412 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:23 smithi047 ceph-mon[39856]: from='mgr.14150 172.21.15.47:0/113394732' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-10T19:01:23.412 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:23 smithi047 ceph-mon[39856]: mgrmap e17: y(active, since 4m), standbys: x 2024-09-10T19:01:23.412 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:23 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/3889635798' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-10T19:01:23.412 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:01:23 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:01:23.083+0000 7f569cb8aec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T19:01:23.412 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:01:23 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:01:23.150+0000 7f569cb8aec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T19:01:23.412 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:01:23 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:01:23.321+0000 7f569cb8aec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T19:01:23.704 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":160994175}]},"active_addr":"172.21.15.47:6800/160994175","active_change":"2024-09-10T18:56:37.929414+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":24115,"name":"x","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.47:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"cephadm","addrvec":[{"type":"v2","addr":"172.21.15.47:0","nonce":1997917857}]},{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.47:0","nonce":143692134}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.47:0","nonce":1269011469}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.47:0","nonce":1348246169}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.47:0","nonce":147696167}]}]} 2024-09-10T19:01:23.708 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-10T19:01:23.709 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-10T19:01:23.709 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid d8563d7e-6fa5-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T19:01:23.967 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:01:24.381 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:01:24 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:01:24.246+0000 7f569cb8aec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T19:01:24.382 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:01:24 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:01:24.316+0000 7f569cb8aec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T19:01:24.592 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:01:24 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:01:24.097+0000 7f00e223dec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T19:01:24.593 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:01:24 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:01:24.162+0000 7f00e223dec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T19:01:25.166 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:01:24 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:01:24.872+0000 7f00e223dec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T19:01:25.166 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:01:24 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:01:24.983+0000 7f00e223dec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T19:01:25.166 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:01:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:01:25.104+0000 7f00e223dec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T19:01:25.166 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:01:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:01:25.163+0000 7f00e223dec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T19:01:25.228 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:01:25 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:01:25.075+0000 7f569cb8aec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T19:01:25.419 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:01:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:01:25.290+0000 7f00e223dec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T19:01:25.419 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:01:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:01:25.359+0000 7f00e223dec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T19:01:25.419 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:01:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:01:25.416+0000 7f00e223dec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T19:01:25.529 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:01:25 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:01:25.226+0000 7f569cb8aec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T19:01:25.530 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:01:25 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:01:25.357+0000 7f569cb8aec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T19:01:25.530 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:01:25 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:01:25.419+0000 7f569cb8aec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T19:01:25.682 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:01:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:01:25.551+0000 7f00e223dec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T19:01:25.682 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:01:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:01:25.616+0000 7f00e223dec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T19:01:25.683 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:01:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:01:25.680+0000 7f00e223dec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T19:01:25.704 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T19:01:25.704 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":61,"fsid":"d8563d7e-6fa5-11ef-bcea-c7b262605968","created":"2024-09-10T18:55:47.433961+0000","modified":"2024-09-10T19:01:03.513820+0000","last_up_change":"2024-09-10T19:00:38.105604+0000","last_in_change":"2024-09-10T19:00:22.059888+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T18:58:48.422291+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":".rgw.root","create_time":"2024-09-10T19:00:49.597078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":3,"pool_name":"datapool","create_time":"2024-09-10T19:00:50.029833+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":57,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-09-10T19:00:51.566234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-09-10T19:00:53.511837+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-09-10T19:00:55.584823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}}}],"osds":[{"osd":0,"uuid":"b4576db6-5c73-4a0e-af6f-6aa568540249","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6801","nonce":840905999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":840905999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":840905999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6803","nonce":840905999}]},"public_addr":"172.21.15.47:6801/840905999","cluster_addr":"172.21.15.47:6802/840905999","heartbeat_back_addr":"172.21.15.47:6804/840905999","heartbeat_front_addr":"172.21.15.47:6803/840905999","state":["exists","up"]},{"osd":1,"uuid":"5ecc9667-1ec5-47d0-b8b0-53411fa13b0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6805","nonce":3075184801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":3075184801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":3075184801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6807","nonce":3075184801}]},"public_addr":"172.21.15.47:6805/3075184801","cluster_addr":"172.21.15.47:6806/3075184801","heartbeat_back_addr":"172.21.15.47:6808/3075184801","heartbeat_front_addr":"172.21.15.47:6807/3075184801","state":["exists","up"]},{"osd":2,"uuid":"68a8c2ac-958c-411c-8a8f-f9e0b453901d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6809","nonce":1234745743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":1234745743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":1234745743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6811","nonce":1234745743}]},"public_addr":"172.21.15.47:6809/1234745743","cluster_addr":"172.21.15.47:6810/1234745743","heartbeat_back_addr":"172.21.15.47:6812/1234745743","heartbeat_front_addr":"172.21.15.47:6811/1234745743","state":["exists","up"]},{"osd":3,"uuid":"126cbc1e-5b4f-439f-9d03-49018692c2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6813","nonce":2685917187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":2685917187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2685917187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6815","nonce":2685917187}]},"public_addr":"172.21.15.47:6813/2685917187","cluster_addr":"172.21.15.47:6814/2685917187","heartbeat_back_addr":"172.21.15.47:6816/2685917187","heartbeat_front_addr":"172.21.15.47:6815/2685917187","state":["exists","up"]},{"osd":4,"uuid":"4a83f8dc-b265-4f60-a8e1-8bcc6fd4130c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":2978707630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6801","nonce":2978707630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6803","nonce":2978707630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":2978707630}]},"public_addr":"172.21.15.149:6800/2978707630","cluster_addr":"172.21.15.149:6801/2978707630","heartbeat_back_addr":"172.21.15.149:6803/2978707630","heartbeat_front_addr":"172.21.15.149:6802/2978707630","state":["exists","up"]},{"osd":5,"uuid":"8ad2f515-f818-40d5-9318-3fb5c932c740","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":902941407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6805","nonce":902941407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6807","nonce":902941407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":902941407}]},"public_addr":"172.21.15.149:6804/902941407","cluster_addr":"172.21.15.149:6805/902941407","heartbeat_back_addr":"172.21.15.149:6807/902941407","heartbeat_front_addr":"172.21.15.149:6806/902941407","state":["exists","up"]},{"osd":6,"uuid":"d7421e61-3716-41c2-862d-8a91c91888e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":525682869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6809","nonce":525682869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6811","nonce":525682869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":525682869}]},"public_addr":"172.21.15.149:6808/525682869","cluster_addr":"172.21.15.149:6809/525682869","heartbeat_back_addr":"172.21.15.149:6811/525682869","heartbeat_front_addr":"172.21.15.149:6810/525682869","state":["exists","up"]},{"osd":7,"uuid":"ad252ba5-666c-46b1-982d-c7e9ffa2b848","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":3375530086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6813","nonce":3375530086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6815","nonce":3375530086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3375530086}]},"public_addr":"172.21.15.149:6812/3375530086","cluster_addr":"172.21.15.149:6813/3375530086","heartbeat_back_addr":"172.21.15.149:6815/3375530086","heartbeat_front_addr":"172.21.15.149:6814/3375530086","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:57:59.034145+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:58:21.972178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:58:44.772844+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:59:07.565542+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:59:28.970709+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:59:50.869973+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T19:00:13.964897+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T19:00:36.809518+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:0/2845803815":"2024-09-11T18:56:37.929313+0000","172.21.15.47:6800/3835757035":"2024-09-11T18:56:15.562858+0000","172.21.15.47:6800/1465212466":"2024-09-11T18:56:37.929313+0000","172.21.15.47:0/351205668":"2024-09-11T18:56:37.929313+0000","172.21.15.47:0/1493757638":"2024-09-11T18:56:15.562858+0000","172.21.15.47:0/438725555":"2024-09-11T18:56:37.929313+0000","172.21.15.47:0/1917819301":"2024-09-11T18:56:15.562858+0000","172.21.15.47:0/1458078728":"2024-09-11T18:56:15.562858+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":2,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T19:01:25.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:25 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/2460498554' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T19:01:25.789 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:25 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/2460498554' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T19:01:25.789 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:01:25 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:01:25.556+0000 7f569cb8aec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T19:01:25.790 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:01:25 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:01:25.627+0000 7f569cb8aec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T19:01:25.790 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:01:25 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:01:25.688+0000 7f569cb8aec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T19:01:26.061 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:01:25 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:01:25.837+0000 7f569cb8aec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T19:01:26.061 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:01:25 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:01:25.904+0000 7f569cb8aec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T19:01:26.061 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:01:25 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:01:25.974+0000 7f569cb8aec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T19:01:26.061 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:01:26 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:01:26.059+0000 7f569cb8aec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T19:01:26.092 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:25 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/2460498554' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T19:01:26.093 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:01:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:01:25.762+0000 7f00e223dec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T19:01:26.093 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:01:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:01:25.819+0000 7f00e223dec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T19:01:26.093 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:01:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:01:25.876+0000 7f00e223dec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T19:01:26.476 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:01:26 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:01:26.120+0000 7f569cb8aec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T19:01:26.476 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:01:26 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:01:26.180+0000 7f569cb8aec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T19:01:26.501 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-10T19:01:26.502 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid d8563d7e-6fa5-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T19:01:26.760 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:01:27.039 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:01:26 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:01:26.989+0000 7f569cb8aec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T19:01:27.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:26 smithi047 ceph-mon[29019]: Standby manager daemon x restarted 2024-09-10T19:01:27.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:26 smithi047 ceph-mon[29019]: Standby manager daemon x started 2024-09-10T19:01:27.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:26 smithi047 ceph-mon[29019]: from='mgr.? 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-10T19:01:27.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:26 smithi047 ceph-mon[29019]: from='mgr.? 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T19:01:27.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:26 smithi047 ceph-mon[29019]: from='mgr.? 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-10T19:01:27.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:26 smithi047 ceph-mon[29019]: from='mgr.? 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T19:01:27.040 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:26 smithi047 ceph-mon[39856]: Standby manager daemon x restarted 2024-09-10T19:01:27.040 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:26 smithi047 ceph-mon[39856]: Standby manager daemon x started 2024-09-10T19:01:27.040 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:26 smithi047 ceph-mon[39856]: from='mgr.? 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-10T19:01:27.041 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:26 smithi047 ceph-mon[39856]: from='mgr.? 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T19:01:27.041 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:26 smithi047 ceph-mon[39856]: from='mgr.? 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-10T19:01:27.041 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:26 smithi047 ceph-mon[39856]: from='mgr.? 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T19:01:27.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:26 smithi149 ceph-mon[33230]: Standby manager daemon x restarted 2024-09-10T19:01:27.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:26 smithi149 ceph-mon[33230]: Standby manager daemon x started 2024-09-10T19:01:27.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:26 smithi149 ceph-mon[33230]: from='mgr.? 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-10T19:01:27.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:26 smithi149 ceph-mon[33230]: from='mgr.? 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T19:01:27.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:26 smithi149 ceph-mon[33230]: from='mgr.? 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-10T19:01:27.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:26 smithi149 ceph-mon[33230]: from='mgr.? 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T19:01:27.093 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:01:26 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:01:26.624+0000 7f00e223dec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T19:01:27.094 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:01:26 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: [10/Sep/2024:19:01:26] ENGINE Bus STARTING 2024-09-10T19:01:27.094 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:01:26 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: CherryPy Checker: 2024-09-10T19:01:27.094 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:01:26 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: The Application mounted at '' has an empty config. 2024-09-10T19:01:27.094 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:01:26 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:01:27.094 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:01:26 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: [10/Sep/2024:19:01:26] ENGINE Serving on http://:::9283 2024-09-10T19:01:27.094 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:01:26 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: [10/Sep/2024:19:01:26] ENGINE Bus STARTED 2024-09-10T19:01:27.353 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:01:27 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: [10/Sep/2024:19:01:27] ENGINE Bus STARTING 2024-09-10T19:01:27.354 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:01:27 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: CherryPy Checker: 2024-09-10T19:01:27.354 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:01:27 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: The Application mounted at '' has an empty config. 2024-09-10T19:01:27.354 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:01:27 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:01:27.354 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:01:27 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: [10/Sep/2024:19:01:27] ENGINE Serving on http://:::9283 2024-09-10T19:01:27.354 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:01:27 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: [10/Sep/2024:19:01:27] ENGINE Bus STARTED 2024-09-10T19:01:28.027 INFO:journalctl@ceph.iscsi.iscsi.a.smithi149.stdout:Sep 10 19:01:28 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-iscsi-iscsi-a[60538]: debug there is no tcmu-runner data available 2024-09-10T19:01:28.028 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:27 smithi149 ceph-mon[33230]: mgrmap e18: y(active, since 4m), standbys: x 2024-09-10T19:01:28.028 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:27 smithi149 ceph-mon[33230]: Active manager daemon y restarted 2024-09-10T19:01:28.028 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:27 smithi149 ceph-mon[33230]: Activating manager daemon y 2024-09-10T19:01:28.028 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:27 smithi149 ceph-mon[33230]: osdmap e62: 8 total, 8 up, 8 in 2024-09-10T19:01:28.028 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:27 smithi149 ceph-mon[33230]: mgrmap e19: y(active, starting, since 0.00942418s), standbys: x 2024-09-10T19:01:28.028 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:27 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T19:01:28.028 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:27 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T19:01:28.028 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:27 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T19:01:28.028 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:27 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T19:01:28.028 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:27 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-10T19:01:28.029 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:27 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T19:01:28.029 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:27 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T19:01:28.029 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:27 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T19:01:28.029 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:27 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T19:01:28.029 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:27 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T19:01:28.029 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:27 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T19:01:28.029 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:27 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T19:01:28.029 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:27 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T19:01:28.029 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:27 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T19:01:28.029 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:27 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T19:01:28.029 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:27 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T19:01:28.029 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:27 smithi149 ceph-mon[33230]: Manager daemon y is now available 2024-09-10T19:01:28.029 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:27 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:01:28.029 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:27 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:01:28.030 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:27 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:01:28.030 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:27 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-10T19:01:28.030 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:27 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-10T19:01:28.039 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:27 smithi047 ceph-mon[29019]: mgrmap e18: y(active, since 4m), standbys: x 2024-09-10T19:01:28.039 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:27 smithi047 ceph-mon[29019]: Active manager daemon y restarted 2024-09-10T19:01:28.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:27 smithi047 ceph-mon[29019]: Activating manager daemon y 2024-09-10T19:01:28.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:27 smithi047 ceph-mon[29019]: osdmap e62: 8 total, 8 up, 8 in 2024-09-10T19:01:28.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:27 smithi047 ceph-mon[29019]: mgrmap e19: y(active, starting, since 0.00942418s), standbys: x 2024-09-10T19:01:28.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:27 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T19:01:28.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:27 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T19:01:28.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:27 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T19:01:28.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:27 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T19:01:28.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:27 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-10T19:01:28.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:27 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T19:01:28.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:27 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T19:01:28.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:27 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T19:01:28.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:27 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T19:01:28.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:27 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T19:01:28.041 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:27 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T19:01:28.041 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:27 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T19:01:28.041 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:27 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T19:01:28.041 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:27 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T19:01:28.041 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:27 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T19:01:28.041 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:27 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T19:01:28.041 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:27 smithi047 ceph-mon[29019]: Manager daemon y is now available 2024-09-10T19:01:28.041 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:27 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:01:28.041 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:27 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:01:28.041 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:27 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:01:28.041 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:27 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-10T19:01:28.041 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:27 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-10T19:01:28.041 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:27 smithi047 ceph-mon[39856]: mgrmap e18: y(active, since 4m), standbys: x 2024-09-10T19:01:28.041 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:27 smithi047 ceph-mon[39856]: Active manager daemon y restarted 2024-09-10T19:01:28.042 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:27 smithi047 ceph-mon[39856]: Activating manager daemon y 2024-09-10T19:01:28.042 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:27 smithi047 ceph-mon[39856]: osdmap e62: 8 total, 8 up, 8 in 2024-09-10T19:01:28.042 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:27 smithi047 ceph-mon[39856]: mgrmap e19: y(active, starting, since 0.00942418s), standbys: x 2024-09-10T19:01:28.042 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:27 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T19:01:28.042 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:27 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T19:01:28.042 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:27 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T19:01:28.042 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:27 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T19:01:28.042 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:27 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-10T19:01:28.042 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:27 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T19:01:28.042 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:27 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T19:01:28.042 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:27 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T19:01:28.042 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:27 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T19:01:28.042 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:27 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T19:01:28.042 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:27 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T19:01:28.042 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:27 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T19:01:28.043 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:27 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T19:01:28.043 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:27 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T19:01:28.043 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:27 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T19:01:28.043 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:27 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T19:01:28.043 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:27 smithi047 ceph-mon[39856]: Manager daemon y is now available 2024-09-10T19:01:28.043 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:27 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:01:28.043 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:27 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:01:28.043 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:27 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:01:28.043 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:27 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-10T19:01:28.043 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:27 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-10T19:01:28.788 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T19:01:28.788 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":62,"fsid":"d8563d7e-6fa5-11ef-bcea-c7b262605968","created":"2024-09-10T18:55:47.433961+0000","modified":"2024-09-10T19:01:26.991757+0000","last_up_change":"2024-09-10T19:00:38.105604+0000","last_in_change":"2024-09-10T19:00:22.059888+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T18:58:48.422291+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":".rgw.root","create_time":"2024-09-10T19:00:49.597078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":3,"pool_name":"datapool","create_time":"2024-09-10T19:00:50.029833+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":57,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-09-10T19:00:51.566234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-09-10T19:00:53.511837+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-09-10T19:00:55.584823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}}}],"osds":[{"osd":0,"uuid":"b4576db6-5c73-4a0e-af6f-6aa568540249","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6801","nonce":840905999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":840905999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":840905999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6803","nonce":840905999}]},"public_addr":"172.21.15.47:6801/840905999","cluster_addr":"172.21.15.47:6802/840905999","heartbeat_back_addr":"172.21.15.47:6804/840905999","heartbeat_front_addr":"172.21.15.47:6803/840905999","state":["exists","up"]},{"osd":1,"uuid":"5ecc9667-1ec5-47d0-b8b0-53411fa13b0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6805","nonce":3075184801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":3075184801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":3075184801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6807","nonce":3075184801}]},"public_addr":"172.21.15.47:6805/3075184801","cluster_addr":"172.21.15.47:6806/3075184801","heartbeat_back_addr":"172.21.15.47:6808/3075184801","heartbeat_front_addr":"172.21.15.47:6807/3075184801","state":["exists","up"]},{"osd":2,"uuid":"68a8c2ac-958c-411c-8a8f-f9e0b453901d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6809","nonce":1234745743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":1234745743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":1234745743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6811","nonce":1234745743}]},"public_addr":"172.21.15.47:6809/1234745743","cluster_addr":"172.21.15.47:6810/1234745743","heartbeat_back_addr":"172.21.15.47:6812/1234745743","heartbeat_front_addr":"172.21.15.47:6811/1234745743","state":["exists","up"]},{"osd":3,"uuid":"126cbc1e-5b4f-439f-9d03-49018692c2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6813","nonce":2685917187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":2685917187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2685917187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6815","nonce":2685917187}]},"public_addr":"172.21.15.47:6813/2685917187","cluster_addr":"172.21.15.47:6814/2685917187","heartbeat_back_addr":"172.21.15.47:6816/2685917187","heartbeat_front_addr":"172.21.15.47:6815/2685917187","state":["exists","up"]},{"osd":4,"uuid":"4a83f8dc-b265-4f60-a8e1-8bcc6fd4130c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":2978707630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6801","nonce":2978707630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6803","nonce":2978707630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":2978707630}]},"public_addr":"172.21.15.149:6800/2978707630","cluster_addr":"172.21.15.149:6801/2978707630","heartbeat_back_addr":"172.21.15.149:6803/2978707630","heartbeat_front_addr":"172.21.15.149:6802/2978707630","state":["exists","up"]},{"osd":5,"uuid":"8ad2f515-f818-40d5-9318-3fb5c932c740","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":902941407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6805","nonce":902941407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6807","nonce":902941407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":902941407}]},"public_addr":"172.21.15.149:6804/902941407","cluster_addr":"172.21.15.149:6805/902941407","heartbeat_back_addr":"172.21.15.149:6807/902941407","heartbeat_front_addr":"172.21.15.149:6806/902941407","state":["exists","up"]},{"osd":6,"uuid":"d7421e61-3716-41c2-862d-8a91c91888e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":525682869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6809","nonce":525682869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6811","nonce":525682869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":525682869}]},"public_addr":"172.21.15.149:6808/525682869","cluster_addr":"172.21.15.149:6809/525682869","heartbeat_back_addr":"172.21.15.149:6811/525682869","heartbeat_front_addr":"172.21.15.149:6810/525682869","state":["exists","up"]},{"osd":7,"uuid":"ad252ba5-666c-46b1-982d-c7e9ffa2b848","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":3375530086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6813","nonce":3375530086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6815","nonce":3375530086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3375530086}]},"public_addr":"172.21.15.149:6812/3375530086","cluster_addr":"172.21.15.149:6813/3375530086","heartbeat_back_addr":"172.21.15.149:6815/3375530086","heartbeat_front_addr":"172.21.15.149:6814/3375530086","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:57:59.034145+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:58:21.972178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:58:44.772844+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:59:07.565542+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:59:28.970709+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:59:50.869973+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T19:00:13.964897+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T19:00:36.809518+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:0/1997917857":"2024-09-11T19:01:26.991709+0000","172.21.15.47:0/1458078728":"2024-09-11T18:56:15.562858+0000","172.21.15.47:0/1917819301":"2024-09-11T18:56:15.562858+0000","172.21.15.47:0/351205668":"2024-09-11T18:56:37.929313+0000","172.21.15.47:0/1493757638":"2024-09-11T18:56:15.562858+0000","172.21.15.47:6800/1465212466":"2024-09-11T18:56:37.929313+0000","172.21.15.47:6800/3835757035":"2024-09-11T18:56:15.562858+0000","172.21.15.47:0/2845803815":"2024-09-11T18:56:37.929313+0000","172.21.15.47:0/143692134":"2024-09-11T19:01:26.991709+0000","172.21.15.47:0/147696167":"2024-09-11T19:01:26.991709+0000","172.21.15.47:0/1269011469":"2024-09-11T19:01:26.991709+0000","172.21.15.47:6800/160994175":"2024-09-11T19:01:26.991709+0000","172.21.15.47:0/438725555":"2024-09-11T18:56:37.929313+0000","172.21.15.47:0/1348246169":"2024-09-11T19:01:26.991709+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T19:01:29.039 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:29 smithi047 ceph-mon[39856]: mgrmap e20: y(active, since 1.01158s), standbys: x 2024-09-10T19:01:29.039 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:29 smithi047 ceph-mon[39856]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:01:29.039 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:29 smithi047 ceph-mon[39856]: pgmap v3: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:01:29.040 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:29 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/2403188235' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T19:01:29.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:29 smithi047 ceph-mon[29019]: mgrmap e20: y(active, since 1.01158s), standbys: x 2024-09-10T19:01:29.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:29 smithi047 ceph-mon[29019]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:01:29.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:29 smithi047 ceph-mon[29019]: pgmap v3: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:01:29.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:29 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/2403188235' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T19:01:29.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:29 smithi149 ceph-mon[33230]: mgrmap e20: y(active, since 1.01158s), standbys: x 2024-09-10T19:01:29.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:29 smithi149 ceph-mon[33230]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:01:29.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:29 smithi149 ceph-mon[33230]: pgmap v3: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:01:29.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:29 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/2403188235' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T19:01:29.449 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid d8563d7e-6fa5-11ef-bcea-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-10T19:01:29.449 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid d8563d7e-6fa5-11ef-bcea-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-10T19:01:29.449 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid d8563d7e-6fa5-11ef-bcea-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-10T19:01:29.450 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid d8563d7e-6fa5-11ef-bcea-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-10T19:01:29.450 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid d8563d7e-6fa5-11ef-bcea-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-10T19:01:29.451 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid d8563d7e-6fa5-11ef-bcea-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-10T19:01:29.451 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid d8563d7e-6fa5-11ef-bcea-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-10T19:01:29.452 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid d8563d7e-6fa5-11ef-bcea-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-10T19:01:29.706 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:01:29 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:01:29.472+0000 7f563b904640 -1 log_channel(cephadm) log [ERR] : [10/Sep/2024:19:01:29] ENGINE Error in HTTPServer.serve 2024-09-10T19:01:29.706 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:01:29 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: Traceback (most recent call last): 2024-09-10T19:01:29.707 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:01:29 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T19:01:29.707 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:01:29 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: self._connections.run(self.expiration_interval) 2024-09-10T19:01:29.707 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:01:29 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T19:01:29.707 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:01:29 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: self._run(expiration_interval) 2024-09-10T19:01:29.707 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:01:29 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T19:01:29.707 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:01:29 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T19:01:29.707 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:01:29 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T19:01:29.707 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:01:29 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T19:01:29.707 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:01:29 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T19:01:29.707 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:01:29 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: s = self.context.wrap_socket( 2024-09-10T19:01:29.707 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:01:29 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T19:01:29.708 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:01:29 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: return self.sslsocket_class._create( 2024-09-10T19:01:29.708 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:01:29 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T19:01:29.708 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:01:29 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: self.do_handshake() 2024-09-10T19:01:29.708 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:01:29 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T19:01:29.708 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:01:29 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: self._sslobj.do_handshake() 2024-09-10T19:01:29.708 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:01:29 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T19:01:29.708 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:01:29 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:01:30.212 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:01:30.214 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:01:30.214 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:01:30.218 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:01:30.267 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:01:30.271 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:01:30.289 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:30 smithi047 ceph-mon[29019]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:01:30.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:30 smithi047 ceph-mon[29019]: mgrmap e21: y(active, since 2s), standbys: x 2024-09-10T19:01:30.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:30 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:01:30.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:30 smithi047 ceph-mon[39856]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:01:30.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:30 smithi047 ceph-mon[39856]: mgrmap e21: y(active, since 2s), standbys: x 2024-09-10T19:01:30.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:30 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:01:30.342 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:30 smithi149 ceph-mon[33230]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:01:30.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:30 smithi149 ceph-mon[33230]: mgrmap e21: y(active, since 2s), standbys: x 2024-09-10T19:01:30.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:30 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:01:31.208 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:31 smithi047 ceph-mon[39856]: [10/Sep/2024:19:01:29] ENGINE Bus STARTING 2024-09-10T19:01:31.208 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:31 smithi047 ceph-mon[39856]: [10/Sep/2024:19:01:29] ENGINE Serving on https://172.21.15.47:7150 2024-09-10T19:01:31.208 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:31 smithi047 ceph-mon[39856]: [10/Sep/2024:19:01:29] ENGINE Bus STARTED 2024-09-10T19:01:31.208 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:31 smithi047 ceph-mon[39856]: [10/Sep/2024:19:01:29] ENGINE Error in HTTPServer.serve 2024-09-10T19:01:31.208 INFO:journalctl@ceph.mon.c.smithi047.stdout: Traceback (most recent call last): 2024-09-10T19:01:31.208 INFO:journalctl@ceph.mon.c.smithi047.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T19:01:31.208 INFO:journalctl@ceph.mon.c.smithi047.stdout: self._connections.run(self.expiration_interval) 2024-09-10T19:01:31.208 INFO:journalctl@ceph.mon.c.smithi047.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T19:01:31.208 INFO:journalctl@ceph.mon.c.smithi047.stdout: self._run(expiration_interval) 2024-09-10T19:01:31.209 INFO:journalctl@ceph.mon.c.smithi047.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T19:01:31.209 INFO:journalctl@ceph.mon.c.smithi047.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T19:01:31.209 INFO:journalctl@ceph.mon.c.smithi047.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T19:01:31.209 INFO:journalctl@ceph.mon.c.smithi047.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T19:01:31.209 INFO:journalctl@ceph.mon.c.smithi047.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T19:01:31.209 INFO:journalctl@ceph.mon.c.smithi047.stdout: s = self.context.wrap_socket( 2024-09-10T19:01:31.209 INFO:journalctl@ceph.mon.c.smithi047.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T19:01:31.209 INFO:journalctl@ceph.mon.c.smithi047.stdout: return self.sslsocket_class._create( 2024-09-10T19:01:31.209 INFO:journalctl@ceph.mon.c.smithi047.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T19:01:31.209 INFO:journalctl@ceph.mon.c.smithi047.stdout: self.do_handshake() 2024-09-10T19:01:31.209 INFO:journalctl@ceph.mon.c.smithi047.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T19:01:31.209 INFO:journalctl@ceph.mon.c.smithi047.stdout: self._sslobj.do_handshake() 2024-09-10T19:01:31.209 INFO:journalctl@ceph.mon.c.smithi047.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T19:01:31.209 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:31 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:01:31.209 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:31 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:01:31.210 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:31 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:01:31.210 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:31 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:01:31.210 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:31 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:01:31.210 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:31 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:01:31.210 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:31 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:01:31.210 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:31 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:01:31.210 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:31 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:01:31.210 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:31 smithi047 ceph-mon[29019]: [10/Sep/2024:19:01:29] ENGINE Bus STARTING 2024-09-10T19:01:31.210 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:31 smithi047 ceph-mon[29019]: [10/Sep/2024:19:01:29] ENGINE Serving on https://172.21.15.47:7150 2024-09-10T19:01:31.210 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:31 smithi047 ceph-mon[29019]: [10/Sep/2024:19:01:29] ENGINE Bus STARTED 2024-09-10T19:01:31.210 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:31 smithi047 ceph-mon[29019]: [10/Sep/2024:19:01:29] ENGINE Error in HTTPServer.serve 2024-09-10T19:01:31.211 INFO:journalctl@ceph.mon.a.smithi047.stdout: Traceback (most recent call last): 2024-09-10T19:01:31.211 INFO:journalctl@ceph.mon.a.smithi047.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T19:01:31.211 INFO:journalctl@ceph.mon.a.smithi047.stdout: self._connections.run(self.expiration_interval) 2024-09-10T19:01:31.211 INFO:journalctl@ceph.mon.a.smithi047.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T19:01:31.211 INFO:journalctl@ceph.mon.a.smithi047.stdout: self._run(expiration_interval) 2024-09-10T19:01:31.211 INFO:journalctl@ceph.mon.a.smithi047.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T19:01:31.211 INFO:journalctl@ceph.mon.a.smithi047.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T19:01:31.211 INFO:journalctl@ceph.mon.a.smithi047.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T19:01:31.211 INFO:journalctl@ceph.mon.a.smithi047.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T19:01:31.211 INFO:journalctl@ceph.mon.a.smithi047.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T19:01:31.211 INFO:journalctl@ceph.mon.a.smithi047.stdout: s = self.context.wrap_socket( 2024-09-10T19:01:31.211 INFO:journalctl@ceph.mon.a.smithi047.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T19:01:31.211 INFO:journalctl@ceph.mon.a.smithi047.stdout: return self.sslsocket_class._create( 2024-09-10T19:01:31.211 INFO:journalctl@ceph.mon.a.smithi047.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T19:01:31.212 INFO:journalctl@ceph.mon.a.smithi047.stdout: self.do_handshake() 2024-09-10T19:01:31.212 INFO:journalctl@ceph.mon.a.smithi047.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T19:01:31.212 INFO:journalctl@ceph.mon.a.smithi047.stdout: self._sslobj.do_handshake() 2024-09-10T19:01:31.212 INFO:journalctl@ceph.mon.a.smithi047.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T19:01:31.212 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:31 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:01:31.212 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:31 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:01:31.212 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:31 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:01:31.212 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:31 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:01:31.212 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:31 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:01:31.212 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:31 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:01:31.212 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:31 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:01:31.212 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:31 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:01:31.213 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:31 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:01:31.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:31 smithi149 ceph-mon[33230]: [10/Sep/2024:19:01:29] ENGINE Bus STARTING 2024-09-10T19:01:31.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:31 smithi149 ceph-mon[33230]: [10/Sep/2024:19:01:29] ENGINE Serving on https://172.21.15.47:7150 2024-09-10T19:01:31.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:31 smithi149 ceph-mon[33230]: [10/Sep/2024:19:01:29] ENGINE Bus STARTED 2024-09-10T19:01:31.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:31 smithi149 ceph-mon[33230]: [10/Sep/2024:19:01:29] ENGINE Error in HTTPServer.serve 2024-09-10T19:01:31.343 INFO:journalctl@ceph.mon.b.smithi149.stdout: Traceback (most recent call last): 2024-09-10T19:01:31.343 INFO:journalctl@ceph.mon.b.smithi149.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T19:01:31.343 INFO:journalctl@ceph.mon.b.smithi149.stdout: self._connections.run(self.expiration_interval) 2024-09-10T19:01:31.343 INFO:journalctl@ceph.mon.b.smithi149.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T19:01:31.344 INFO:journalctl@ceph.mon.b.smithi149.stdout: self._run(expiration_interval) 2024-09-10T19:01:31.344 INFO:journalctl@ceph.mon.b.smithi149.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T19:01:31.344 INFO:journalctl@ceph.mon.b.smithi149.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T19:01:31.344 INFO:journalctl@ceph.mon.b.smithi149.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T19:01:31.344 INFO:journalctl@ceph.mon.b.smithi149.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T19:01:31.344 INFO:journalctl@ceph.mon.b.smithi149.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T19:01:31.344 INFO:journalctl@ceph.mon.b.smithi149.stdout: s = self.context.wrap_socket( 2024-09-10T19:01:31.344 INFO:journalctl@ceph.mon.b.smithi149.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T19:01:31.344 INFO:journalctl@ceph.mon.b.smithi149.stdout: return self.sslsocket_class._create( 2024-09-10T19:01:31.344 INFO:journalctl@ceph.mon.b.smithi149.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T19:01:31.344 INFO:journalctl@ceph.mon.b.smithi149.stdout: self.do_handshake() 2024-09-10T19:01:31.344 INFO:journalctl@ceph.mon.b.smithi149.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T19:01:31.345 INFO:journalctl@ceph.mon.b.smithi149.stdout: self._sslobj.do_handshake() 2024-09-10T19:01:31.345 INFO:journalctl@ceph.mon.b.smithi149.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T19:01:31.345 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:31 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:01:31.345 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:31 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:01:31.345 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:31 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:01:31.345 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:31 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:01:31.345 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:31 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:01:31.345 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:31 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:01:31.345 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:31 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:01:31.345 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:31 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:01:31.345 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:31 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:01:32.284 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:32 smithi047 ceph-mon[29019]: Adjusting osd_memory_target on smithi149 to 941.8M 2024-09-10T19:01:32.285 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:32 smithi047 ceph-mon[29019]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:01:32.285 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:32 smithi047 ceph-mon[29019]: mgrmap e22: y(active, since 4s), standbys: x 2024-09-10T19:01:32.285 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:32 smithi047 ceph-mon[39856]: Adjusting osd_memory_target on smithi149 to 941.8M 2024-09-10T19:01:32.285 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:32 smithi047 ceph-mon[39856]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:01:32.285 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:32 smithi047 ceph-mon[39856]: mgrmap e22: y(active, since 4s), standbys: x 2024-09-10T19:01:32.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:32 smithi149 ceph-mon[33230]: Adjusting osd_memory_target on smithi149 to 941.8M 2024-09-10T19:01:32.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:32 smithi149 ceph-mon[33230]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:01:32.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:32 smithi149 ceph-mon[33230]: mgrmap e22: y(active, since 4s), standbys: x 2024-09-10T19:01:33.342 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:01:32 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: ::ffff:172.21.15.149 - - [10/Sep/2024:19:01:32] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:01:34.342 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:34 smithi149 ceph-mon[33230]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1 KiB/s rd, 1 op/s 2024-09-10T19:01:34.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:34 smithi047 ceph-mon[29019]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1 KiB/s rd, 1 op/s 2024-09-10T19:01:34.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:34 smithi047 ceph-mon[39856]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1 KiB/s rd, 1 op/s 2024-09-10T19:01:36.141 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:36 smithi047 ceph-mon[29019]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-10T19:01:36.141 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:36 smithi047 ceph-mon[39856]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-10T19:01:36.342 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:36 smithi149 ceph-mon[33230]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-10T19:01:37.231 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:01:38.134 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:38 smithi047 ceph-mon[39856]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-10T19:01:38.134 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:38 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:01:38.135 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:38 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:01:38.135 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:38 smithi047 ceph-mon[29019]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-10T19:01:38.135 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:38 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:01:38.135 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:38 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:01:38.342 INFO:journalctl@ceph.iscsi.iscsi.a.smithi149.stdout:Sep 10 19:01:38 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-iscsi-iscsi-a[60538]: debug there is no tcmu-runner data available 2024-09-10T19:01:38.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:38 smithi149 ceph-mon[33230]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-10T19:01:38.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:38 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:01:38.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:38 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:01:38.473 INFO:teuthology.orchestra.run.smithi047.stdout:158913789973 2024-09-10T19:01:38.473 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid d8563d7e-6fa5-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-10T19:01:39.289 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:39 smithi047 ceph-mon[29019]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:01:39.289 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:39 smithi047 ceph-mon[39856]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:01:39.592 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:39 smithi149 ceph-mon[33230]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:01:40.353 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:40 smithi047 ceph-mon[29019]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 17 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-10T19:01:40.353 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:40 smithi047 ceph-mon[39856]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 17 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-10T19:01:40.381 INFO:teuthology.orchestra.run.smithi047.stdout:184683593745 2024-09-10T19:01:40.381 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid d8563d7e-6fa5-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-10T19:01:40.592 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:40 smithi149 ceph-mon[33230]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 17 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-10T19:01:40.984 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:01:41.661 INFO:teuthology.orchestra.run.smithi047.stdout:34359738415 2024-09-10T19:01:41.662 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid d8563d7e-6fa5-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-10T19:01:42.289 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:42 smithi047 ceph-mon[29019]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 16 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-10T19:01:42.289 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:42 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:01:42.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:42 smithi047 ceph-mon[39856]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 16 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-10T19:01:42.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:42 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:01:42.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:42 smithi149 ceph-mon[33230]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 16 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-10T19:01:42.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:42 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:01:42.807 INFO:teuthology.orchestra.run.smithi047.stdout:77309411365 2024-09-10T19:01:42.807 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid d8563d7e-6fa5-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-10T19:01:42.960 INFO:teuthology.orchestra.run.smithi047.stdout:55834574889 2024-09-10T19:01:42.960 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid d8563d7e-6fa5-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-10T19:01:43.342 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:01:42 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: ::ffff:172.21.15.149 - - [10/Sep/2024:19:01:42] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:01:43.847 INFO:teuthology.orchestra.run.smithi047.stdout:210453397518 2024-09-10T19:01:43.848 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid d8563d7e-6fa5-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-10T19:01:44.083 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:01:44.094 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:43 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:01:44.094 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:43 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:01:44.094 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:43 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:01:44.094 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:43 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:01:44.094 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:43 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:01:44.094 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:43 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:01:44.094 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:43 smithi047 ceph-mon[29019]: Adjusting osd_memory_target on smithi047 to 3761M 2024-09-10T19:01:44.094 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:43 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:01:44.094 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:43 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:01:44.094 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:43 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:01:44.095 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:43 smithi047 ceph-mon[29019]: Updating smithi047:/etc/ceph/ceph.conf 2024-09-10T19:01:44.095 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:43 smithi047 ceph-mon[29019]: Updating smithi149:/etc/ceph/ceph.conf 2024-09-10T19:01:44.095 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:43 smithi047 ceph-mon[29019]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 16 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-10T19:01:44.095 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:43 smithi047 ceph-mon[29019]: Updating smithi149:/var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T19:01:44.095 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:43 smithi047 ceph-mon[29019]: Updating smithi047:/var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T19:01:44.095 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:43 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:01:44.095 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:43 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:01:44.095 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:43 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:01:44.095 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:43 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:01:44.095 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:43 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:01:44.095 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:43 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:01:44.095 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:43 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:01:44.095 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:43 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:01:44.096 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:43 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:01:44.096 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:43 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:01:44.096 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:43 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:01:44.096 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:43 smithi047 ceph-mon[39856]: Adjusting osd_memory_target on smithi047 to 3761M 2024-09-10T19:01:44.096 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:43 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:01:44.096 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:43 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:01:44.096 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:43 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:01:44.096 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:43 smithi047 ceph-mon[39856]: Updating smithi047:/etc/ceph/ceph.conf 2024-09-10T19:01:44.096 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:43 smithi047 ceph-mon[39856]: Updating smithi149:/etc/ceph/ceph.conf 2024-09-10T19:01:44.096 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:43 smithi047 ceph-mon[39856]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 16 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-10T19:01:44.096 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:43 smithi047 ceph-mon[39856]: Updating smithi149:/var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T19:01:44.096 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:43 smithi047 ceph-mon[39856]: Updating smithi047:/var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T19:01:44.096 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:43 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:01:44.096 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:43 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:01:44.096 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:43 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:01:44.097 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:43 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:01:44.097 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:43 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:01:44.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:43 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:01:44.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:43 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:01:44.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:43 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:01:44.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:43 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:01:44.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:43 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:01:44.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:43 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:01:44.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:43 smithi149 ceph-mon[33230]: Adjusting osd_memory_target on smithi047 to 3761M 2024-09-10T19:01:44.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:43 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:01:44.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:43 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:01:44.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:43 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:01:44.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:43 smithi149 ceph-mon[33230]: Updating smithi047:/etc/ceph/ceph.conf 2024-09-10T19:01:44.344 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:43 smithi149 ceph-mon[33230]: Updating smithi149:/etc/ceph/ceph.conf 2024-09-10T19:01:44.344 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:43 smithi149 ceph-mon[33230]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 16 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-10T19:01:44.344 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:43 smithi149 ceph-mon[33230]: Updating smithi149:/var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T19:01:44.344 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:43 smithi149 ceph-mon[33230]: Updating smithi047:/var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T19:01:44.344 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:43 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:01:44.344 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:43 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:01:44.344 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:43 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:01:44.344 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:43 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:01:44.344 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:43 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:01:45.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:44 smithi047 ceph-mon[29019]: Updating smithi149:/etc/ceph/ceph.client.admin.keyring 2024-09-10T19:01:45.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:44 smithi047 ceph-mon[29019]: Updating smithi047:/etc/ceph/ceph.client.admin.keyring 2024-09-10T19:01:45.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:44 smithi047 ceph-mon[29019]: Updating smithi149:/var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T19:01:45.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:44 smithi047 ceph-mon[29019]: Updating smithi047:/var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T19:01:45.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:44 smithi047 ceph-mon[29019]: Deploying daemon alertmanager.a on smithi047 2024-09-10T19:01:45.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:45 smithi047 ceph-mon[39856]: Updating smithi149:/etc/ceph/ceph.client.admin.keyring 2024-09-10T19:01:45.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:45 smithi047 ceph-mon[39856]: Updating smithi047:/etc/ceph/ceph.client.admin.keyring 2024-09-10T19:01:45.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:45 smithi047 ceph-mon[39856]: Updating smithi149:/var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T19:01:45.291 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:45 smithi047 ceph-mon[39856]: Updating smithi047:/var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T19:01:45.291 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:45 smithi047 ceph-mon[39856]: Deploying daemon alertmanager.a on smithi047 2024-09-10T19:01:45.342 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:45 smithi149 ceph-mon[33230]: Updating smithi149:/etc/ceph/ceph.client.admin.keyring 2024-09-10T19:01:45.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:45 smithi149 ceph-mon[33230]: Updating smithi047:/etc/ceph/ceph.client.admin.keyring 2024-09-10T19:01:45.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:45 smithi149 ceph-mon[33230]: Updating smithi149:/var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T19:01:45.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:45 smithi149 ceph-mon[33230]: Updating smithi047:/var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T19:01:45.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:45 smithi149 ceph-mon[33230]: Deploying daemon alertmanager.a on smithi047 2024-09-10T19:01:46.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:46 smithi047 ceph-mon[29019]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-10T19:01:46.275 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:46 smithi047 ceph-mon[39856]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-10T19:01:46.342 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:46 smithi149 ceph-mon[33230]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-10T19:01:46.588 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:01:47.909 INFO:teuthology.orchestra.run.smithi047.stdout:184683593747 2024-09-10T19:01:48.289 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:48 smithi047 ceph-mon[29019]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:01:48.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:48 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/935110782' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-10T19:01:48.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:48 smithi047 ceph-mon[39856]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:01:48.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:48 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/935110782' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-10T19:01:48.342 INFO:journalctl@ceph.iscsi.iscsi.a.smithi149.stdout:Sep 10 19:01:48 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-iscsi-iscsi-a[60538]: debug there is no tcmu-runner data available 2024-09-10T19:01:48.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:48 smithi149 ceph-mon[33230]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:01:48.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:48 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/935110782' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-10T19:01:48.604 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:01:48.623 INFO:tasks.cephadm.ceph_manager.ceph:need seq 184683593745 got 184683593747 for osd.6 2024-09-10T19:01:48.624 DEBUG:teuthology.parallel:result is None 2024-09-10T19:01:49.212 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:49 smithi047 ceph-mon[29019]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:01:49.212 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:49 smithi047 ceph-mon[39856]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:01:49.217 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:01:49.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:49 smithi149 ceph-mon[33230]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:01:50.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:50 smithi047 ceph-mon[29019]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:01:50.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:50 smithi047 ceph-mon[39856]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:01:50.342 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:50 smithi149 ceph-mon[33230]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:01:50.489 INFO:teuthology.orchestra.run.smithi047.stdout:133143986206 2024-09-10T19:01:50.489 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid d8563d7e-6fa5-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-10T19:01:51.117 INFO:teuthology.orchestra.run.smithi047.stdout:111669149729 2024-09-10T19:01:51.117 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid d8563d7e-6fa5-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-10T19:01:51.781 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:01:51.781 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:01:52.289 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:52 smithi047 ceph-mon[29019]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:01:52.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:52 smithi047 ceph-mon[39856]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:01:52.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:52 smithi149 ceph-mon[33230]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:01:53.343 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:01:52 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: ::ffff:172.21.15.149 - - [10/Sep/2024:19:01:52] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:01:54.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:54 smithi149 ceph-mon[33230]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T19:01:54.449 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:54 smithi047 ceph-mon[29019]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T19:01:54.450 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:54 smithi047 ceph-mon[39856]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T19:01:55.301 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:01:56.342 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:56 smithi149 ceph-mon[33230]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:01:56.425 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:56 smithi047 ceph-mon[29019]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:01:56.425 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:56 smithi047 ceph-mon[39856]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:01:57.225 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:57 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:01:57.226 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:57 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:01:57.342 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:57 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:01:58.226 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:58 smithi047 ceph-mon[29019]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:01:58.226 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:58 smithi047 ceph-mon[39856]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:01:58.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:58 smithi149 ceph-mon[33230]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:01:58.343 INFO:journalctl@ceph.iscsi.iscsi.a.smithi149.stdout:Sep 10 19:01:58 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-iscsi-iscsi-a[60538]: debug there is no tcmu-runner data available 2024-09-10T19:01:58.926 INFO:teuthology.orchestra.run.smithi047.stdout:34359738418 2024-09-10T19:01:59.342 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:59 smithi149 ceph-mon[33230]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:01:59.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:01:59 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/3145868703' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-10T19:01:59.539 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:59 smithi047 ceph-mon[39856]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:01:59.539 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:01:59 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/3145868703' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-10T19:01:59.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:59 smithi047 ceph-mon[29019]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:01:59.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:01:59 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/3145868703' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-10T19:02:00.342 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:00 smithi149 ceph-mon[33230]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:02:00.349 INFO:teuthology.orchestra.run.smithi047.stdout:77309411369 2024-09-10T19:02:00.349 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:00 smithi047 ceph-mon[29019]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:02:00.350 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:00 smithi047 ceph-mon[39856]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:02:01.196 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:01 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/3671224100' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-10T19:02:01.196 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:01 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/3671224100' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-10T19:02:01.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:01 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/3671224100' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-10T19:02:01.618 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411365 got 77309411369 for osd.2 2024-09-10T19:02:01.619 DEBUG:teuthology.parallel:result is None 2024-09-10T19:02:02.411 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:02 smithi047 ceph-mon[29019]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:02:02.411 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:02 smithi047 ceph-mon[39856]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:02:02.418 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:02:02.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:02 smithi149 ceph-mon[33230]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:02:03.342 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:02:02 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: ::ffff:172.21.15.149 - - [10/Sep/2024:19:02:02] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:02:04.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:04 smithi047 ceph-mon[29019]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:02:04.539 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:04 smithi047 ceph-mon[39856]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:02:04.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:04 smithi149 ceph-mon[33230]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:02:05.129 INFO:teuthology.orchestra.run.smithi047.stdout:158913789979 2024-09-10T19:02:05.332 INFO:teuthology.orchestra.run.smithi047.stdout:210453397523 2024-09-10T19:02:05.986 INFO:tasks.cephadm.ceph_manager.ceph:need seq 210453397518 got 210453397523 for osd.7 2024-09-10T19:02:05.986 DEBUG:teuthology.parallel:result is None 2024-09-10T19:02:06.217 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:06 smithi047 ceph-mon[29019]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:02:06.217 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:06 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/1761279755' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-10T19:02:06.217 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:06 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/2093035282' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-10T19:02:06.217 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:06 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/3513274984' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-10T19:02:06.217 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:06 smithi047 ceph-mon[39856]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:02:06.217 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:06 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/1761279755' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-10T19:02:06.217 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:06 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/2093035282' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-10T19:02:06.217 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:06 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/3513274984' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-10T19:02:06.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:06 smithi149 ceph-mon[33230]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:02:06.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:06 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/1761279755' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-10T19:02:06.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:06 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/2093035282' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-10T19:02:06.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:06 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/3513274984' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-10T19:02:06.621 INFO:tasks.cephadm.ceph_manager.ceph:need seq 158913789973 got 158913789979 for osd.5 2024-09-10T19:02:06.621 DEBUG:teuthology.parallel:result is None 2024-09-10T19:02:06.738 INFO:teuthology.orchestra.run.smithi047.stdout:133143986209 2024-09-10T19:02:07.630 INFO:teuthology.orchestra.run.smithi047.stdout:55834574894 2024-09-10T19:02:07.820 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738415 got 34359738418 for osd.0 2024-09-10T19:02:07.820 DEBUG:teuthology.parallel:result is None 2024-09-10T19:02:07.937 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986206 got 133143986209 for osd.4 2024-09-10T19:02:07.937 DEBUG:teuthology.parallel:result is None 2024-09-10T19:02:08.157 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:08 smithi047 ceph-mon[39856]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:02:08.157 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:08 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/1117687080' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-10T19:02:08.157 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:08 smithi047 ceph-mon[29019]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:02:08.157 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:08 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/1117687080' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-10T19:02:08.343 INFO:journalctl@ceph.iscsi.iscsi.a.smithi149.stdout:Sep 10 19:02:08 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-iscsi-iscsi-a[60538]: debug there is no tcmu-runner data available 2024-09-10T19:02:08.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:08 smithi149 ceph-mon[33230]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:02:08.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:08 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/1117687080' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-10T19:02:09.175 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:09 smithi047 ceph-mon[29019]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:02:09.176 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:09 smithi047 ceph-mon[39856]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:02:09.177 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574889 got 55834574894 for osd.1 2024-09-10T19:02:09.177 DEBUG:teuthology.parallel:result is None 2024-09-10T19:02:09.592 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:09 smithi149 ceph-mon[33230]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:02:10.190 INFO:teuthology.orchestra.run.smithi047.stdout:111669149733 2024-09-10T19:02:10.539 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:10 smithi047 ceph-mon[39856]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:02:10.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:10 smithi047 ceph-mon[29019]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:02:10.592 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:10 smithi149 ceph-mon[33230]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:02:10.927 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149729 got 111669149733 for osd.3 2024-09-10T19:02:10.927 DEBUG:teuthology.parallel:result is None 2024-09-10T19:02:10.927 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-10T19:02:10.927 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid d8563d7e-6fa5-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T19:02:11.197 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:02:11.286 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:11 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/2767665174' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-10T19:02:11.287 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:11 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/2767665174' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-10T19:02:11.287 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Sep 10 19:02:11 smithi047 systemd[1]: Started Ceph alertmanager.a for d8563d7e-6fa5-11ef-bcea-c7b262605968. 2024-09-10T19:02:11.539 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:02:11 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: [10/Sep/2024:19:02:11] ENGINE Bus STOPPING 2024-09-10T19:02:11.592 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:11 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/2767665174' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-10T19:02:11.843 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:02:11 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: [10/Sep/2024:19:02:11] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-10T19:02:11.844 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:02:11 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: [10/Sep/2024:19:02:11] ENGINE Bus STOPPED 2024-09-10T19:02:11.844 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:02:11 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: [10/Sep/2024:19:02:11] ENGINE Bus STARTING 2024-09-10T19:02:11.844 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:02:11 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: [10/Sep/2024:19:02:11] ENGINE Serving on http://:::9283 2024-09-10T19:02:11.844 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:02:11 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: [10/Sep/2024:19:02:11] ENGINE Bus STARTED 2024-09-10T19:02:12.473 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:12 smithi047 ceph-mon[39856]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:02:12.473 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:12 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:12.473 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:12 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:12.473 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:12 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:12.473 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:12 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:12.473 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:12 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:12.474 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:12 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:12.474 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:12 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-10T19:02:12.474 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:12 smithi047 ceph-mon[39856]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-10T19:02:12.474 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:12 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:12.474 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:12 smithi047 ceph-mon[39856]: Deploying daemon grafana.a on smithi149 2024-09-10T19:02:12.474 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:12 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:12.474 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:12 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:02:12.474 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:12 smithi047 ceph-mon[29019]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:02:12.474 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:12 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:12.474 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:12 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:12.475 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:12 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:12.475 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:12 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:12.475 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:12 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:12.475 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:12 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:12.475 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:12 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-10T19:02:12.475 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:12 smithi047 ceph-mon[29019]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-10T19:02:12.475 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:12 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:12.475 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:12 smithi047 ceph-mon[29019]: Deploying daemon grafana.a on smithi149 2024-09-10T19:02:12.475 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:12 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:12.475 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:12 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:02:12.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:12 smithi149 ceph-mon[33230]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:02:12.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:12 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:12.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:12 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:12.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:12 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:12.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:12 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:12.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:12 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:12.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:12 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:12.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:12 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-10T19:02:12.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:12 smithi149 ceph-mon[33230]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-10T19:02:12.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:12 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:12.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:12 smithi149 ceph-mon[33230]: Deploying daemon grafana.a on smithi149 2024-09-10T19:02:12.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:12 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:12.594 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:12 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:02:12.994 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T19:02:12.996 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-09-10T19:02:13.289 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Sep 10 19:02:13 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-alertmanager-a[72004]: ts=2024-09-10T19:02:13.043Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000348595s 2024-09-10T19:02:13.342 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:02:12 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: ::ffff:172.21.15.149 - - [10/Sep/2024:19:02:12] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:02:13.647 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2024-09-10T19:02:11.016911+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596188,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":923,"num_read_kb":780,"num_write":543,"num_write_kb":1671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":548,"ondisk_log_size":548,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2385452,"kb_used_data":6612,"kb_used_omap":13,"kb_used_meta":2378738,"kb_avail":747575764,"statfs":{"total":767960285184,"available":765517582336,"internally_reserved":0,"allocated":6770688,"data_stored":3650184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2435828416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002747"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.581570+0000","last_change":"2024-09-10T19:00:57.180451+0000","last_active":"2024-09-10T19:01:03.581570+0000","last_peered":"2024-09-10T19:01:03.581570+0000","last_clean":"2024-09-10T19:01:03.581570+0000","last_became_active":"2024-09-10T19:00:57.180062+0000","last_became_peered":"2024-09-10T19:00:57.180062+0000","last_unstale":"2024-09-10T19:01:03.581570+0000","last_undegraded":"2024-09-10T19:01:03.581570+0000","last_fullsized":"2024-09-10T19:01:03.581570+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_clean_scrub_stamp":"2024-09-10T19:00:56.075080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:01:34.222831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.519310+0000","last_change":"2024-09-10T19:00:51.060398+0000","last_active":"2024-09-10T19:01:03.519310+0000","last_peered":"2024-09-10T19:01:03.519310+0000","last_clean":"2024-09-10T19:01:03.519310+0000","last_became_active":"2024-09-10T19:00:51.059990+0000","last_became_peered":"2024-09-10T19:00:51.059990+0000","last_unstale":"2024-09-10T19:01:03.519310+0000","last_undegraded":"2024-09-10T19:01:03.519310+0000","last_fullsized":"2024-09-10T19:01:03.519310+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_clean_scrub_stamp":"2024-09-10T19:00:50.042140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:12:18.449280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"60'15","reported_seq":41,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.581544+0000","last_change":"2024-09-10T19:00:53.073694+0000","last_active":"2024-09-10T19:01:03.581544+0000","last_peered":"2024-09-10T19:01:03.581544+0000","last_clean":"2024-09-10T19:01:03.581544+0000","last_became_active":"2024-09-10T19:00:53.073198+0000","last_became_peered":"2024-09-10T19:00:53.073198+0000","last_unstale":"2024-09-10T19:01:03.581544+0000","last_undegraded":"2024-09-10T19:01:03.581544+0000","last_fullsized":"2024-09-10T19:01:03.581544+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_clean_scrub_stamp":"2024-09-10T19:00:52.048288+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:16:51.164807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.517187+0000","last_change":"2024-09-10T19:00:55.094707+0000","last_active":"2024-09-10T19:01:03.517187+0000","last_peered":"2024-09-10T19:01:03.517187+0000","last_clean":"2024-09-10T19:01:03.517187+0000","last_became_active":"2024-09-10T19:00:55.094430+0000","last_became_peered":"2024-09-10T19:00:55.094430+0000","last_unstale":"2024-09-10T19:01:03.517187+0000","last_undegraded":"2024-09-10T19:01:03.517187+0000","last_fullsized":"2024-09-10T19:01:03.517187+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_clean_scrub_stamp":"2024-09-10T19:00:54.064181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:23:57.287401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.582831+0000","last_change":"2024-09-10T19:00:51.062082+0000","last_active":"2024-09-10T19:01:03.582831+0000","last_peered":"2024-09-10T19:01:03.582831+0000","last_clean":"2024-09-10T19:01:03.582831+0000","last_became_active":"2024-09-10T19:00:51.061543+0000","last_became_peered":"2024-09-10T19:00:51.061543+0000","last_unstale":"2024-09-10T19:01:03.582831+0000","last_undegraded":"2024-09-10T19:01:03.582831+0000","last_fullsized":"2024-09-10T19:01:03.582831+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_clean_scrub_stamp":"2024-09-10T19:00:50.042140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:41:42.232976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"60'9","reported_seq":32,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.517195+0000","last_change":"2024-09-10T19:00:53.063307+0000","last_active":"2024-09-10T19:01:03.517195+0000","last_peered":"2024-09-10T19:01:03.517195+0000","last_clean":"2024-09-10T19:01:03.517195+0000","last_became_active":"2024-09-10T19:00:53.063011+0000","last_became_peered":"2024-09-10T19:00:53.063011+0000","last_unstale":"2024-09-10T19:01:03.517195+0000","last_undegraded":"2024-09-10T19:01:03.517195+0000","last_fullsized":"2024-09-10T19:01:03.517195+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_clean_scrub_stamp":"2024-09-10T19:00:52.048288+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:45:31.862691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.581272+0000","last_change":"2024-09-10T19:00:55.092719+0000","last_active":"2024-09-10T19:01:03.581272+0000","last_peered":"2024-09-10T19:01:03.581272+0000","last_clean":"2024-09-10T19:01:03.581272+0000","last_became_active":"2024-09-10T19:00:55.092395+0000","last_became_peered":"2024-09-10T19:00:55.092395+0000","last_unstale":"2024-09-10T19:01:03.581272+0000","last_undegraded":"2024-09-10T19:01:03.581272+0000","last_fullsized":"2024-09-10T19:01:03.581272+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_clean_scrub_stamp":"2024-09-10T19:00:54.064181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:33:52.953786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.517336+0000","last_change":"2024-09-10T19:00:57.097214+0000","last_active":"2024-09-10T19:01:03.517336+0000","last_peered":"2024-09-10T19:01:03.517336+0000","last_clean":"2024-09-10T19:01:03.517336+0000","last_became_active":"2024-09-10T19:00:57.096877+0000","last_became_peered":"2024-09-10T19:00:57.096877+0000","last_unstale":"2024-09-10T19:01:03.517336+0000","last_undegraded":"2024-09-10T19:01:03.517336+0000","last_fullsized":"2024-09-10T19:01:03.517336+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_clean_scrub_stamp":"2024-09-10T19:00:56.075080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:04:45.819611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.582864+0000","last_change":"2024-09-10T19:00:51.066666+0000","last_active":"2024-09-10T19:01:03.582864+0000","last_peered":"2024-09-10T19:01:03.582864+0000","last_clean":"2024-09-10T19:01:03.582864+0000","last_became_active":"2024-09-10T19:00:51.060459+0000","last_became_peered":"2024-09-10T19:00:51.060459+0000","last_unstale":"2024-09-10T19:01:03.582864+0000","last_undegraded":"2024-09-10T19:01:03.582864+0000","last_fullsized":"2024-09-10T19:01:03.582864+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_clean_scrub_stamp":"2024-09-10T19:00:50.042140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:58.199866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"60'5","reported_seq":26,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.517349+0000","last_change":"2024-09-10T19:00:53.069292+0000","last_active":"2024-09-10T19:01:03.517349+0000","last_peered":"2024-09-10T19:01:03.517349+0000","last_clean":"2024-09-10T19:01:03.517349+0000","last_became_active":"2024-09-10T19:00:53.068926+0000","last_became_peered":"2024-09-10T19:00:53.068926+0000","last_unstale":"2024-09-10T19:01:03.517349+0000","last_undegraded":"2024-09-10T19:01:03.517349+0000","last_fullsized":"2024-09-10T19:01:03.517349+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_clean_scrub_stamp":"2024-09-10T19:00:52.048288+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:29:46.049574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.581266+0000","last_change":"2024-09-10T19:00:55.090773+0000","last_active":"2024-09-10T19:01:03.581266+0000","last_peered":"2024-09-10T19:01:03.581266+0000","last_clean":"2024-09-10T19:01:03.581266+0000","last_became_active":"2024-09-10T19:00:55.088329+0000","last_became_peered":"2024-09-10T19:00:55.088329+0000","last_unstale":"2024-09-10T19:01:03.581266+0000","last_undegraded":"2024-09-10T19:01:03.581266+0000","last_fullsized":"2024-09-10T19:01:03.581266+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_clean_scrub_stamp":"2024-09-10T19:00:54.064181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:34:20.701575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.582391+0000","last_change":"2024-09-10T19:00:57.101639+0000","last_active":"2024-09-10T19:01:03.582391+0000","last_peered":"2024-09-10T19:01:03.582391+0000","last_clean":"2024-09-10T19:01:03.582391+0000","last_became_active":"2024-09-10T19:00:57.101289+0000","last_became_peered":"2024-09-10T19:00:57.101289+0000","last_unstale":"2024-09-10T19:01:03.582391+0000","last_undegraded":"2024-09-10T19:01:03.582391+0000","last_fullsized":"2024-09-10T19:01:03.582391+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_clean_scrub_stamp":"2024-09-10T19:00:56.075080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:18:11.861317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.582872+0000","last_change":"2024-09-10T19:00:51.068359+0000","last_active":"2024-09-10T19:01:03.582872+0000","last_peered":"2024-09-10T19:01:03.582872+0000","last_clean":"2024-09-10T19:01:03.582872+0000","last_became_active":"2024-09-10T19:00:51.067717+0000","last_became_peered":"2024-09-10T19:00:51.067717+0000","last_unstale":"2024-09-10T19:01:03.582872+0000","last_undegraded":"2024-09-10T19:01:03.582872+0000","last_fullsized":"2024-09-10T19:01:03.582872+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_clean_scrub_stamp":"2024-09-10T19:00:50.042140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:51:43.094466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"60'9","reported_seq":32,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.517646+0000","last_change":"2024-09-10T19:00:53.065093+0000","last_active":"2024-09-10T19:01:03.517646+0000","last_peered":"2024-09-10T19:01:03.517646+0000","last_clean":"2024-09-10T19:01:03.517646+0000","last_became_active":"2024-09-10T19:00:53.064756+0000","last_became_peered":"2024-09-10T19:00:53.064756+0000","last_unstale":"2024-09-10T19:01:03.517646+0000","last_undegraded":"2024-09-10T19:01:03.517646+0000","last_fullsized":"2024-09-10T19:01:03.517646+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_clean_scrub_stamp":"2024-09-10T19:00:52.048288+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:34:56.308621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.582376+0000","last_change":"2024-09-10T19:00:55.089110+0000","last_active":"2024-09-10T19:01:03.582376+0000","last_peered":"2024-09-10T19:01:03.582376+0000","last_clean":"2024-09-10T19:01:03.582376+0000","last_became_active":"2024-09-10T19:00:55.088796+0000","last_became_peered":"2024-09-10T19:00:55.088796+0000","last_unstale":"2024-09-10T19:01:03.582376+0000","last_undegraded":"2024-09-10T19:01:03.582376+0000","last_fullsized":"2024-09-10T19:01:03.582376+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_clean_scrub_stamp":"2024-09-10T19:00:54.064181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:56.957040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.516997+0000","last_change":"2024-09-10T19:00:57.098397+0000","last_active":"2024-09-10T19:01:03.516997+0000","last_peered":"2024-09-10T19:01:03.516997+0000","last_clean":"2024-09-10T19:01:03.516997+0000","last_became_active":"2024-09-10T19:00:57.097566+0000","last_became_peered":"2024-09-10T19:00:57.097566+0000","last_unstale":"2024-09-10T19:01:03.516997+0000","last_undegraded":"2024-09-10T19:01:03.516997+0000","last_fullsized":"2024-09-10T19:01:03.516997+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_clean_scrub_stamp":"2024-09-10T19:00:56.075080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:45.230498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.d","version":"60'17","reported_seq":44,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.517903+0000","last_change":"2024-09-10T19:00:53.069427+0000","last_active":"2024-09-10T19:01:03.517903+0000","last_peered":"2024-09-10T19:01:03.517903+0000","last_clean":"2024-09-10T19:01:03.517903+0000","last_became_active":"2024-09-10T19:00:53.068732+0000","last_became_peered":"2024-09-10T19:00:53.068732+0000","last_unstale":"2024-09-10T19:01:03.517903+0000","last_undegraded":"2024-09-10T19:01:03.517903+0000","last_fullsized":"2024-09-10T19:01:03.517903+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_clean_scrub_stamp":"2024-09-10T19:00:52.048288+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:25:30.165923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.582341+0000","last_change":"2024-09-10T19:00:51.060295+0000","last_active":"2024-09-10T19:01:03.582341+0000","last_peered":"2024-09-10T19:01:03.582341+0000","last_clean":"2024-09-10T19:01:03.582341+0000","last_became_active":"2024-09-10T19:00:51.059971+0000","last_became_peered":"2024-09-10T19:00:51.059971+0000","last_unstale":"2024-09-10T19:01:03.582341+0000","last_undegraded":"2024-09-10T19:01:03.582341+0000","last_fullsized":"2024-09-10T19:01:03.582341+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_clean_scrub_stamp":"2024-09-10T19:00:50.042140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:28.889400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.582545+0000","last_change":"2024-09-10T19:00:55.088922+0000","last_active":"2024-09-10T19:01:03.582545+0000","last_peered":"2024-09-10T19:01:03.582545+0000","last_clean":"2024-09-10T19:01:03.582545+0000","last_became_active":"2024-09-10T19:00:55.088601+0000","last_became_peered":"2024-09-10T19:00:55.088601+0000","last_unstale":"2024-09-10T19:01:03.582545+0000","last_undegraded":"2024-09-10T19:01:03.582545+0000","last_fullsized":"2024-09-10T19:01:03.582545+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_clean_scrub_stamp":"2024-09-10T19:00:54.064181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:57.963253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.519192+0000","last_change":"2024-09-10T19:00:57.088212+0000","last_active":"2024-09-10T19:01:03.519192+0000","last_peered":"2024-09-10T19:01:03.519192+0000","last_clean":"2024-09-10T19:01:03.519192+0000","last_became_active":"2024-09-10T19:00:57.087890+0000","last_became_peered":"2024-09-10T19:00:57.087890+0000","last_unstale":"2024-09-10T19:01:03.519192+0000","last_undegraded":"2024-09-10T19:01:03.519192+0000","last_fullsized":"2024-09-10T19:01:03.519192+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_clean_scrub_stamp":"2024-09-10T19:00:56.075080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:37:05.603419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"60'10","reported_seq":31,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.517585+0000","last_change":"2024-09-10T19:00:53.065010+0000","last_active":"2024-09-10T19:01:03.517585+0000","last_peered":"2024-09-10T19:01:03.517585+0000","last_clean":"2024-09-10T19:01:03.517585+0000","last_became_active":"2024-09-10T19:00:53.064689+0000","last_became_peered":"2024-09-10T19:00:53.064689+0000","last_unstale":"2024-09-10T19:01:03.517585+0000","last_undegraded":"2024-09-10T19:01:03.517585+0000","last_fullsized":"2024-09-10T19:01:03.517585+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_clean_scrub_stamp":"2024-09-10T19:00:52.048288+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:53:17.508552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.581949+0000","last_change":"2024-09-10T19:00:51.062914+0000","last_active":"2024-09-10T19:01:03.581949+0000","last_peered":"2024-09-10T19:01:03.581949+0000","last_clean":"2024-09-10T19:01:03.581949+0000","last_became_active":"2024-09-10T19:00:51.060081+0000","last_became_peered":"2024-09-10T19:00:51.060081+0000","last_unstale":"2024-09-10T19:01:03.581949+0000","last_undegraded":"2024-09-10T19:01:03.581949+0000","last_fullsized":"2024-09-10T19:01:03.581949+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_clean_scrub_stamp":"2024-09-10T19:00:50.042140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.738297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"60'8","reported_seq":37,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T19:01:59.170495+0000","last_change":"2024-09-10T19:00:55.088579+0000","last_active":"2024-09-10T19:01:59.170495+0000","last_peered":"2024-09-10T19:01:59.170495+0000","last_clean":"2024-09-10T19:01:59.170495+0000","last_became_active":"2024-09-10T19:00:55.088301+0000","last_became_peered":"2024-09-10T19:00:55.088301+0000","last_unstale":"2024-09-10T19:01:59.170495+0000","last_undegraded":"2024-09-10T19:01:59.170495+0000","last_fullsized":"2024-09-10T19:01:59.170495+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_clean_scrub_stamp":"2024-09-10T19:00:54.064181+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:38:26.702904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.517691+0000","last_change":"2024-09-10T19:00:57.095572+0000","last_active":"2024-09-10T19:01:03.517691+0000","last_peered":"2024-09-10T19:01:03.517691+0000","last_clean":"2024-09-10T19:01:03.517691+0000","last_became_active":"2024-09-10T19:00:57.095309+0000","last_became_peered":"2024-09-10T19:00:57.095309+0000","last_unstale":"2024-09-10T19:01:03.517691+0000","last_undegraded":"2024-09-10T19:01:03.517691+0000","last_fullsized":"2024-09-10T19:01:03.517691+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_clean_scrub_stamp":"2024-09-10T19:00:56.075080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:12:20.359347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"60'15","reported_seq":41,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.583136+0000","last_change":"2024-09-10T19:00:53.075783+0000","last_active":"2024-09-10T19:01:03.583136+0000","last_peered":"2024-09-10T19:01:03.583136+0000","last_clean":"2024-09-10T19:01:03.583136+0000","last_became_active":"2024-09-10T19:00:53.075350+0000","last_became_peered":"2024-09-10T19:00:53.075350+0000","last_unstale":"2024-09-10T19:01:03.583136+0000","last_undegraded":"2024-09-10T19:01:03.583136+0000","last_fullsized":"2024-09-10T19:01:03.583136+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_clean_scrub_stamp":"2024-09-10T19:00:52.048288+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:06:42.843477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.581700+0000","last_change":"2024-09-10T19:00:51.062932+0000","last_active":"2024-09-10T19:01:03.581700+0000","last_peered":"2024-09-10T19:01:03.581700+0000","last_clean":"2024-09-10T19:01:03.581700+0000","last_became_active":"2024-09-10T19:00:51.062715+0000","last_became_peered":"2024-09-10T19:00:51.062715+0000","last_unstale":"2024-09-10T19:01:03.581700+0000","last_undegraded":"2024-09-10T19:01:03.581700+0000","last_fullsized":"2024-09-10T19:01:03.581700+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_clean_scrub_stamp":"2024-09-10T19:00:50.042140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:24:19.538737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"60'8","reported_seq":37,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T19:01:59.170777+0000","last_change":"2024-09-10T19:00:55.095285+0000","last_active":"2024-09-10T19:01:59.170777+0000","last_peered":"2024-09-10T19:01:59.170777+0000","last_clean":"2024-09-10T19:01:59.170777+0000","last_became_active":"2024-09-10T19:00:55.093624+0000","last_became_peered":"2024-09-10T19:00:55.093624+0000","last_unstale":"2024-09-10T19:01:59.170777+0000","last_undegraded":"2024-09-10T19:01:59.170777+0000","last_fullsized":"2024-09-10T19:01:59.170777+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_clean_scrub_stamp":"2024-09-10T19:00:54.064181+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:38:06.008272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.582575+0000","last_change":"2024-09-10T19:00:57.100829+0000","last_active":"2024-09-10T19:01:03.582575+0000","last_peered":"2024-09-10T19:01:03.582575+0000","last_clean":"2024-09-10T19:01:03.582575+0000","last_became_active":"2024-09-10T19:00:57.100557+0000","last_became_peered":"2024-09-10T19:00:57.100557+0000","last_unstale":"2024-09-10T19:01:03.582575+0000","last_undegraded":"2024-09-10T19:01:03.582575+0000","last_fullsized":"2024-09-10T19:01:03.582575+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_clean_scrub_stamp":"2024-09-10T19:00:56.075080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:33:00.791836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"60'11","reported_seq":35,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.517791+0000","last_change":"2024-09-10T19:00:53.065614+0000","last_active":"2024-09-10T19:01:03.517791+0000","last_peered":"2024-09-10T19:01:03.517791+0000","last_clean":"2024-09-10T19:01:03.517791+0000","last_became_active":"2024-09-10T19:00:53.065329+0000","last_became_peered":"2024-09-10T19:00:53.065329+0000","last_unstale":"2024-09-10T19:01:03.517791+0000","last_undegraded":"2024-09-10T19:01:03.517791+0000","last_fullsized":"2024-09-10T19:01:03.517791+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_clean_scrub_stamp":"2024-09-10T19:00:52.048288+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:12:25.850396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.581309+0000","last_change":"2024-09-10T19:00:51.066299+0000","last_active":"2024-09-10T19:01:03.581309+0000","last_peered":"2024-09-10T19:01:03.581309+0000","last_clean":"2024-09-10T19:01:03.581309+0000","last_became_active":"2024-09-10T19:00:51.066028+0000","last_became_peered":"2024-09-10T19:00:51.066028+0000","last_unstale":"2024-09-10T19:01:03.581309+0000","last_undegraded":"2024-09-10T19:01:03.581309+0000","last_fullsized":"2024-09-10T19:01:03.581309+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_clean_scrub_stamp":"2024-09-10T19:00:50.042140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:55:51.248136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.582650+0000","last_change":"2024-09-10T19:00:55.087250+0000","last_active":"2024-09-10T19:01:03.582650+0000","last_peered":"2024-09-10T19:01:03.582650+0000","last_clean":"2024-09-10T19:01:03.582650+0000","last_became_active":"2024-09-10T19:00:55.086777+0000","last_became_peered":"2024-09-10T19:00:55.086777+0000","last_unstale":"2024-09-10T19:01:03.582650+0000","last_undegraded":"2024-09-10T19:01:03.582650+0000","last_fullsized":"2024-09-10T19:01:03.582650+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_clean_scrub_stamp":"2024-09-10T19:00:54.064181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:46:08.875218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.582146+0000","last_change":"2024-09-10T19:00:57.178915+0000","last_active":"2024-09-10T19:01:03.582146+0000","last_peered":"2024-09-10T19:01:03.582146+0000","last_clean":"2024-09-10T19:01:03.582146+0000","last_became_active":"2024-09-10T19:00:57.178295+0000","last_became_peered":"2024-09-10T19:00:57.178295+0000","last_unstale":"2024-09-10T19:01:03.582146+0000","last_undegraded":"2024-09-10T19:01:03.582146+0000","last_fullsized":"2024-09-10T19:01:03.582146+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_clean_scrub_stamp":"2024-09-10T19:00:56.075080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:29:19.875609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"60'14","reported_seq":37,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.517748+0000","last_change":"2024-09-10T19:00:53.063992+0000","last_active":"2024-09-10T19:01:03.517748+0000","last_peered":"2024-09-10T19:01:03.517748+0000","last_clean":"2024-09-10T19:01:03.517748+0000","last_became_active":"2024-09-10T19:00:53.063647+0000","last_became_peered":"2024-09-10T19:00:53.063647+0000","last_unstale":"2024-09-10T19:01:03.517748+0000","last_undegraded":"2024-09-10T19:01:03.517748+0000","last_fullsized":"2024-09-10T19:01:03.517748+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_clean_scrub_stamp":"2024-09-10T19:00:52.048288+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:11:45.504877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.582221+0000","last_change":"2024-09-10T19:00:51.064084+0000","last_active":"2024-09-10T19:01:03.582221+0000","last_peered":"2024-09-10T19:01:03.582221+0000","last_clean":"2024-09-10T19:01:03.582221+0000","last_became_active":"2024-09-10T19:00:51.060461+0000","last_became_peered":"2024-09-10T19:00:51.060461+0000","last_unstale":"2024-09-10T19:01:03.582221+0000","last_undegraded":"2024-09-10T19:01:03.582221+0000","last_fullsized":"2024-09-10T19:01:03.582221+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_clean_scrub_stamp":"2024-09-10T19:00:50.042140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:39:16.021756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"60'8","reported_seq":37,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T19:01:59.170577+0000","last_change":"2024-09-10T19:00:55.088897+0000","last_active":"2024-09-10T19:01:59.170577+0000","last_peered":"2024-09-10T19:01:59.170577+0000","last_clean":"2024-09-10T19:01:59.170577+0000","last_became_active":"2024-09-10T19:00:55.088594+0000","last_became_peered":"2024-09-10T19:00:55.088594+0000","last_unstale":"2024-09-10T19:01:59.170577+0000","last_undegraded":"2024-09-10T19:01:59.170577+0000","last_fullsized":"2024-09-10T19:01:59.170577+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_clean_scrub_stamp":"2024-09-10T19:00:54.064181+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:21.399086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.581560+0000","last_change":"2024-09-10T19:00:57.178594+0000","last_active":"2024-09-10T19:01:03.581560+0000","last_peered":"2024-09-10T19:01:03.581560+0000","last_clean":"2024-09-10T19:01:03.581560+0000","last_became_active":"2024-09-10T19:00:57.178291+0000","last_became_peered":"2024-09-10T19:00:57.178291+0000","last_unstale":"2024-09-10T19:01:03.581560+0000","last_undegraded":"2024-09-10T19:01:03.581560+0000","last_fullsized":"2024-09-10T19:01:03.581560+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_clean_scrub_stamp":"2024-09-10T19:00:56.075080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:32:44.542113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"60'18","reported_seq":48,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.581255+0000","last_change":"2024-09-10T19:00:53.525001+0000","last_active":"2024-09-10T19:01:03.581255+0000","last_peered":"2024-09-10T19:01:03.581255+0000","last_clean":"2024-09-10T19:01:03.581255+0000","last_became_active":"2024-09-10T19:00:53.524689+0000","last_became_peered":"2024-09-10T19:00:53.524689+0000","last_unstale":"2024-09-10T19:01:03.581255+0000","last_undegraded":"2024-09-10T19:01:03.581255+0000","last_fullsized":"2024-09-10T19:01:03.581255+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_clean_scrub_stamp":"2024-09-10T19:00:52.048288+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:17:53.807969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.583022+0000","last_change":"2024-09-10T19:00:51.063467+0000","last_active":"2024-09-10T19:01:03.583022+0000","last_peered":"2024-09-10T19:01:03.583022+0000","last_clean":"2024-09-10T19:01:03.583022+0000","last_became_active":"2024-09-10T19:00:51.063210+0000","last_became_peered":"2024-09-10T19:00:51.063210+0000","last_unstale":"2024-09-10T19:01:03.583022+0000","last_undegraded":"2024-09-10T19:01:03.583022+0000","last_fullsized":"2024-09-10T19:01:03.583022+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_clean_scrub_stamp":"2024-09-10T19:00:50.042140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:51.264503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.517263+0000","last_change":"2024-09-10T19:00:55.090004+0000","last_active":"2024-09-10T19:01:03.517263+0000","last_peered":"2024-09-10T19:01:03.517263+0000","last_clean":"2024-09-10T19:01:03.517263+0000","last_became_active":"2024-09-10T19:00:55.089535+0000","last_became_peered":"2024-09-10T19:00:55.089535+0000","last_unstale":"2024-09-10T19:01:03.517263+0000","last_undegraded":"2024-09-10T19:01:03.517263+0000","last_fullsized":"2024-09-10T19:01:03.517263+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_clean_scrub_stamp":"2024-09-10T19:00:54.064181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:29:02.951124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.517723+0000","last_change":"2024-09-10T19:00:57.087842+0000","last_active":"2024-09-10T19:01:03.517723+0000","last_peered":"2024-09-10T19:01:03.517723+0000","last_clean":"2024-09-10T19:01:03.517723+0000","last_became_active":"2024-09-10T19:00:57.087580+0000","last_became_peered":"2024-09-10T19:00:57.087580+0000","last_unstale":"2024-09-10T19:01:03.517723+0000","last_undegraded":"2024-09-10T19:01:03.517723+0000","last_fullsized":"2024-09-10T19:01:03.517723+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_clean_scrub_stamp":"2024-09-10T19:00:56.075080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:17.300293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"60'19","reported_seq":52,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.517739+0000","last_change":"2024-09-10T19:00:53.523951+0000","last_active":"2024-09-10T19:01:03.517739+0000","last_peered":"2024-09-10T19:01:03.517739+0000","last_clean":"2024-09-10T19:01:03.517739+0000","last_became_active":"2024-09-10T19:00:53.523676+0000","last_became_peered":"2024-09-10T19:00:53.523676+0000","last_unstale":"2024-09-10T19:01:03.517739+0000","last_undegraded":"2024-09-10T19:01:03.517739+0000","last_fullsized":"2024-09-10T19:01:03.517739+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_clean_scrub_stamp":"2024-09-10T19:00:52.048288+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:13:19.024014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.582930+0000","last_change":"2024-09-10T19:00:51.057836+0000","last_active":"2024-09-10T19:01:03.582930+0000","last_peered":"2024-09-10T19:01:03.582930+0000","last_clean":"2024-09-10T19:01:03.582930+0000","last_became_active":"2024-09-10T19:00:51.057446+0000","last_became_peered":"2024-09-10T19:00:51.057446+0000","last_unstale":"2024-09-10T19:01:03.582930+0000","last_undegraded":"2024-09-10T19:01:03.582930+0000","last_fullsized":"2024-09-10T19:01:03.582930+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_clean_scrub_stamp":"2024-09-10T19:00:50.042140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:16:43.741187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.581290+0000","last_change":"2024-09-10T19:00:55.086455+0000","last_active":"2024-09-10T19:01:03.581290+0000","last_peered":"2024-09-10T19:01:03.581290+0000","last_clean":"2024-09-10T19:01:03.581290+0000","last_became_active":"2024-09-10T19:00:55.086145+0000","last_became_peered":"2024-09-10T19:00:55.086145+0000","last_unstale":"2024-09-10T19:01:03.581290+0000","last_undegraded":"2024-09-10T19:01:03.581290+0000","last_fullsized":"2024-09-10T19:01:03.581290+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_clean_scrub_stamp":"2024-09-10T19:00:54.064181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:23:56.102156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.581187+0000","last_change":"2024-09-10T19:00:57.178515+0000","last_active":"2024-09-10T19:01:03.581187+0000","last_peered":"2024-09-10T19:01:03.581187+0000","last_clean":"2024-09-10T19:01:03.581187+0000","last_became_active":"2024-09-10T19:00:57.178200+0000","last_became_peered":"2024-09-10T19:00:57.178200+0000","last_unstale":"2024-09-10T19:01:03.581187+0000","last_undegraded":"2024-09-10T19:01:03.581187+0000","last_fullsized":"2024-09-10T19:01:03.581187+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_clean_scrub_stamp":"2024-09-10T19:00:56.075080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:40:54.285437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"60'10","reported_seq":31,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.581799+0000","last_change":"2024-09-10T19:00:53.066199+0000","last_active":"2024-09-10T19:01:03.581799+0000","last_peered":"2024-09-10T19:01:03.581799+0000","last_clean":"2024-09-10T19:01:03.581799+0000","last_became_active":"2024-09-10T19:00:53.065660+0000","last_became_peered":"2024-09-10T19:00:53.065660+0000","last_unstale":"2024-09-10T19:01:03.581799+0000","last_undegraded":"2024-09-10T19:01:03.581799+0000","last_fullsized":"2024-09-10T19:01:03.581799+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_clean_scrub_stamp":"2024-09-10T19:00:52.048288+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:31.996329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.582791+0000","last_change":"2024-09-10T19:00:51.063105+0000","last_active":"2024-09-10T19:01:03.582791+0000","last_peered":"2024-09-10T19:01:03.582791+0000","last_clean":"2024-09-10T19:01:03.582791+0000","last_became_active":"2024-09-10T19:00:51.059864+0000","last_became_peered":"2024-09-10T19:00:51.059864+0000","last_unstale":"2024-09-10T19:01:03.582791+0000","last_undegraded":"2024-09-10T19:01:03.582791+0000","last_fullsized":"2024-09-10T19:01:03.582791+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_clean_scrub_stamp":"2024-09-10T19:00:50.042140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:01:14.619190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"60'8","reported_seq":37,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T19:01:59.170495+0000","last_change":"2024-09-10T19:00:55.089934+0000","last_active":"2024-09-10T19:01:59.170495+0000","last_peered":"2024-09-10T19:01:59.170495+0000","last_clean":"2024-09-10T19:01:59.170495+0000","last_became_active":"2024-09-10T19:00:55.089650+0000","last_became_peered":"2024-09-10T19:00:55.089650+0000","last_unstale":"2024-09-10T19:01:59.170495+0000","last_undegraded":"2024-09-10T19:01:59.170495+0000","last_fullsized":"2024-09-10T19:01:59.170495+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_clean_scrub_stamp":"2024-09-10T19:00:54.064181+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:03:36.427766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.516817+0000","last_change":"2024-09-10T19:00:57.098224+0000","last_active":"2024-09-10T19:01:03.516817+0000","last_peered":"2024-09-10T19:01:03.516817+0000","last_clean":"2024-09-10T19:01:03.516817+0000","last_became_active":"2024-09-10T19:00:57.096762+0000","last_became_peered":"2024-09-10T19:00:57.096762+0000","last_unstale":"2024-09-10T19:01:03.516817+0000","last_undegraded":"2024-09-10T19:01:03.516817+0000","last_fullsized":"2024-09-10T19:01:03.516817+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_clean_scrub_stamp":"2024-09-10T19:00:56.075080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:42.237137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"60'28","reported_seq":79,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T19:01:59.170522+0000","last_change":"2024-09-10T19:00:53.068106+0000","last_active":"2024-09-10T19:01:59.170522+0000","last_peered":"2024-09-10T19:01:59.170522+0000","last_clean":"2024-09-10T19:01:59.170522+0000","last_became_active":"2024-09-10T19:00:53.067827+0000","last_became_peered":"2024-09-10T19:00:53.067827+0000","last_unstale":"2024-09-10T19:01:59.170522+0000","last_undegraded":"2024-09-10T19:01:59.170522+0000","last_fullsized":"2024-09-10T19:01:59.170522+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_clean_scrub_stamp":"2024-09-10T19:00:52.048288+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:39:17.494401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.582440+0000","last_change":"2024-09-10T19:00:51.065887+0000","last_active":"2024-09-10T19:01:03.582440+0000","last_peered":"2024-09-10T19:01:03.582440+0000","last_clean":"2024-09-10T19:01:03.582440+0000","last_became_active":"2024-09-10T19:00:51.065539+0000","last_became_peered":"2024-09-10T19:00:51.065539+0000","last_unstale":"2024-09-10T19:01:03.582440+0000","last_undegraded":"2024-09-10T19:01:03.582440+0000","last_fullsized":"2024-09-10T19:01:03.582440+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_clean_scrub_stamp":"2024-09-10T19:00:50.042140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:09:49.598245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.518999+0000","last_change":"2024-09-10T19:00:55.089168+0000","last_active":"2024-09-10T19:01:03.518999+0000","last_peered":"2024-09-10T19:01:03.518999+0000","last_clean":"2024-09-10T19:01:03.518999+0000","last_became_active":"2024-09-10T19:00:55.088776+0000","last_became_peered":"2024-09-10T19:00:55.088776+0000","last_unstale":"2024-09-10T19:01:03.518999+0000","last_undegraded":"2024-09-10T19:01:03.518999+0000","last_fullsized":"2024-09-10T19:01:03.518999+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_clean_scrub_stamp":"2024-09-10T19:00:54.064181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:09:03.566361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"60'1","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.582681+0000","last_change":"2024-09-10T19:00:57.093578+0000","last_active":"2024-09-10T19:01:03.582681+0000","last_peered":"2024-09-10T19:01:03.582681+0000","last_clean":"2024-09-10T19:01:03.582681+0000","last_became_active":"2024-09-10T19:00:57.093224+0000","last_became_peered":"2024-09-10T19:00:57.093224+0000","last_unstale":"2024-09-10T19:01:03.582681+0000","last_undegraded":"2024-09-10T19:01:03.582681+0000","last_fullsized":"2024-09-10T19:01:03.582681+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_clean_scrub_stamp":"2024-09-10T19:00:56.075080+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:33:32.775617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"60'13","reported_seq":43,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.583148+0000","last_change":"2024-09-10T19:00:53.075018+0000","last_active":"2024-09-10T19:01:03.583148+0000","last_peered":"2024-09-10T19:01:03.583148+0000","last_clean":"2024-09-10T19:01:03.583148+0000","last_became_active":"2024-09-10T19:00:53.073402+0000","last_became_peered":"2024-09-10T19:00:53.073402+0000","last_unstale":"2024-09-10T19:01:03.583148+0000","last_undegraded":"2024-09-10T19:01:03.583148+0000","last_fullsized":"2024-09-10T19:01:03.583148+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_clean_scrub_stamp":"2024-09-10T19:00:52.048288+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:12:30.101521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"60'5","reported_seq":96,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T19:02:06.246157+0000","last_change":"2024-09-10T19:00:55.085761+0000","last_active":"2024-09-10T19:02:06.246157+0000","last_peered":"2024-09-10T19:02:06.246157+0000","last_clean":"2024-09-10T19:02:06.246157+0000","last_became_active":"2024-09-10T19:00:51.054621+0000","last_became_peered":"2024-09-10T19:00:51.054621+0000","last_unstale":"2024-09-10T19:02:06.246157+0000","last_undegraded":"2024-09-10T19:02:06.246157+0000","last_fullsized":"2024-09-10T19:02:06.246157+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_clean_scrub_stamp":"2024-09-10T19:00:50.042140+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:07:10.164529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000292179,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":70,"num_read_kb":65,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.518179+0000","last_change":"2024-09-10T19:00:51.058933+0000","last_active":"2024-09-10T19:01:03.518179+0000","last_peered":"2024-09-10T19:01:03.518179+0000","last_clean":"2024-09-10T19:01:03.518179+0000","last_became_active":"2024-09-10T19:00:51.058651+0000","last_became_peered":"2024-09-10T19:00:51.058651+0000","last_unstale":"2024-09-10T19:01:03.518179+0000","last_undegraded":"2024-09-10T19:01:03.518179+0000","last_fullsized":"2024-09-10T19:01:03.518179+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_clean_scrub_stamp":"2024-09-10T19:00:50.042140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:28:10.445664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.517376+0000","last_change":"2024-09-10T19:00:55.084077+0000","last_active":"2024-09-10T19:01:03.517376+0000","last_peered":"2024-09-10T19:01:03.517376+0000","last_clean":"2024-09-10T19:01:03.517376+0000","last_became_active":"2024-09-10T19:00:55.083721+0000","last_became_peered":"2024-09-10T19:00:55.083721+0000","last_unstale":"2024-09-10T19:01:03.517376+0000","last_undegraded":"2024-09-10T19:01:03.517376+0000","last_fullsized":"2024-09-10T19:01:03.517376+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_clean_scrub_stamp":"2024-09-10T19:00:54.064181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:16:38.135924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.582474+0000","last_change":"2024-09-10T19:00:57.179341+0000","last_active":"2024-09-10T19:01:03.582474+0000","last_peered":"2024-09-10T19:01:03.582474+0000","last_clean":"2024-09-10T19:01:03.582474+0000","last_became_active":"2024-09-10T19:00:57.178744+0000","last_became_peered":"2024-09-10T19:00:57.178744+0000","last_unstale":"2024-09-10T19:01:03.582474+0000","last_undegraded":"2024-09-10T19:01:03.582474+0000","last_fullsized":"2024-09-10T19:01:03.582474+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_clean_scrub_stamp":"2024-09-10T19:00:56.075080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:30:59.675745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"60'12","reported_seq":34,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.517657+0000","last_change":"2024-09-10T19:00:53.069777+0000","last_active":"2024-09-10T19:01:03.517657+0000","last_peered":"2024-09-10T19:01:03.517657+0000","last_clean":"2024-09-10T19:01:03.517657+0000","last_became_active":"2024-09-10T19:00:53.069483+0000","last_became_peered":"2024-09-10T19:00:53.069483+0000","last_unstale":"2024-09-10T19:01:03.517657+0000","last_undegraded":"2024-09-10T19:01:03.517657+0000","last_fullsized":"2024-09-10T19:01:03.517657+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_clean_scrub_stamp":"2024-09-10T19:00:52.048288+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:02:29.102995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"55'1","reported_seq":24,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.517523+0000","last_change":"2024-09-10T19:00:55.081265+0000","last_active":"2024-09-10T19:01:03.517523+0000","last_peered":"2024-09-10T19:01:03.517523+0000","last_clean":"2024-09-10T19:01:03.517523+0000","last_became_active":"2024-09-10T19:00:51.060401+0000","last_became_peered":"2024-09-10T19:00:51.060401+0000","last_unstale":"2024-09-10T19:01:03.517523+0000","last_undegraded":"2024-09-10T19:01:03.517523+0000","last_fullsized":"2024-09-10T19:01:03.517523+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_clean_scrub_stamp":"2024-09-10T19:00:50.042140+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:23:02.165568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015209259999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.581376+0000","last_change":"2024-09-10T19:00:51.066604+0000","last_active":"2024-09-10T19:01:03.581376+0000","last_peered":"2024-09-10T19:01:03.581376+0000","last_clean":"2024-09-10T19:01:03.581376+0000","last_became_active":"2024-09-10T19:00:51.066342+0000","last_became_peered":"2024-09-10T19:00:51.066342+0000","last_unstale":"2024-09-10T19:01:03.581376+0000","last_undegraded":"2024-09-10T19:01:03.581376+0000","last_fullsized":"2024-09-10T19:01:03.581376+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_clean_scrub_stamp":"2024-09-10T19:00:50.042140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:25:59.085736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.582731+0000","last_change":"2024-09-10T19:00:55.094027+0000","last_active":"2024-09-10T19:01:03.582731+0000","last_peered":"2024-09-10T19:01:03.582731+0000","last_clean":"2024-09-10T19:01:03.582731+0000","last_became_active":"2024-09-10T19:00:55.093729+0000","last_became_peered":"2024-09-10T19:00:55.093729+0000","last_unstale":"2024-09-10T19:01:03.582731+0000","last_undegraded":"2024-09-10T19:01:03.582731+0000","last_fullsized":"2024-09-10T19:01:03.582731+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_clean_scrub_stamp":"2024-09-10T19:00:54.064181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:48:45.962070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.582474+0000","last_change":"2024-09-10T19:00:57.099929+0000","last_active":"2024-09-10T19:01:03.582474+0000","last_peered":"2024-09-10T19:01:03.582474+0000","last_clean":"2024-09-10T19:01:03.582474+0000","last_became_active":"2024-09-10T19:00:57.099659+0000","last_became_peered":"2024-09-10T19:00:57.099659+0000","last_unstale":"2024-09-10T19:01:03.582474+0000","last_undegraded":"2024-09-10T19:01:03.582474+0000","last_fullsized":"2024-09-10T19:01:03.582474+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_clean_scrub_stamp":"2024-09-10T19:00:56.075080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:08:43.226734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"60'14","reported_seq":38,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.582235+0000","last_change":"2024-09-10T19:00:53.524620+0000","last_active":"2024-09-10T19:01:03.582235+0000","last_peered":"2024-09-10T19:01:03.582235+0000","last_clean":"2024-09-10T19:01:03.582235+0000","last_became_active":"2024-09-10T19:00:53.524335+0000","last_became_peered":"2024-09-10T19:00:53.524335+0000","last_unstale":"2024-09-10T19:01:03.582235+0000","last_undegraded":"2024-09-10T19:01:03.582235+0000","last_fullsized":"2024-09-10T19:01:03.582235+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_clean_scrub_stamp":"2024-09-10T19:00:52.048288+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:05:09.467617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"57'2","reported_seq":25,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.581993+0000","last_change":"2024-09-10T19:00:55.079489+0000","last_active":"2024-09-10T19:01:03.581993+0000","last_peered":"2024-09-10T19:01:03.581993+0000","last_clean":"2024-09-10T19:01:03.581993+0000","last_became_active":"2024-09-10T19:00:51.060879+0000","last_became_peered":"2024-09-10T19:00:51.060879+0000","last_unstale":"2024-09-10T19:01:03.581993+0000","last_undegraded":"2024-09-10T19:01:03.581993+0000","last_fullsized":"2024-09-10T19:01:03.581993+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_clean_scrub_stamp":"2024-09-10T19:00:50.042140+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:53:08.915514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030390389999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.582467+0000","last_change":"2024-09-10T19:00:51.063012+0000","last_active":"2024-09-10T19:01:03.582467+0000","last_peered":"2024-09-10T19:01:03.582467+0000","last_clean":"2024-09-10T19:01:03.582467+0000","last_became_active":"2024-09-10T19:00:51.062733+0000","last_became_peered":"2024-09-10T19:00:51.062733+0000","last_unstale":"2024-09-10T19:01:03.582467+0000","last_undegraded":"2024-09-10T19:01:03.582467+0000","last_fullsized":"2024-09-10T19:01:03.582467+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_clean_scrub_stamp":"2024-09-10T19:00:50.042140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:32:42.120072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"62'79","reported_seq":52,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.058735+0000","last_change":"2024-09-10T19:00:39.512067+0000","last_active":"2024-09-10T19:01:29.058735+0000","last_peered":"2024-09-10T19:01:29.058735+0000","last_clean":"2024-09-10T19:01:29.058735+0000","last_became_active":"2024-09-10T19:00:39.505499+0000","last_became_peered":"2024-09-10T19:00:39.505499+0000","last_unstale":"2024-09-10T19:01:29.058735+0000","last_undegraded":"2024-09-10T19:01:29.058735+0000","last_fullsized":"2024-09-10T19:01:29.058735+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:48.679084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:58:48.679084+0000","last_clean_scrub_stamp":"2024-09-10T18:58:48.679084+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:04:33.625322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.582917+0000","last_change":"2024-09-10T19:00:55.084167+0000","last_active":"2024-09-10T19:01:03.582917+0000","last_peered":"2024-09-10T19:01:03.582917+0000","last_clean":"2024-09-10T19:01:03.582917+0000","last_became_active":"2024-09-10T19:00:55.083860+0000","last_became_peered":"2024-09-10T19:00:55.083860+0000","last_unstale":"2024-09-10T19:01:03.582917+0000","last_undegraded":"2024-09-10T19:01:03.582917+0000","last_fullsized":"2024-09-10T19:01:03.582917+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_clean_scrub_stamp":"2024-09-10T19:00:54.064181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:04:42.181455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.582877+0000","last_change":"2024-09-10T19:00:57.101521+0000","last_active":"2024-09-10T19:01:03.582877+0000","last_peered":"2024-09-10T19:01:03.582877+0000","last_clean":"2024-09-10T19:01:03.582877+0000","last_became_active":"2024-09-10T19:00:57.101236+0000","last_became_peered":"2024-09-10T19:00:57.101236+0000","last_unstale":"2024-09-10T19:01:03.582877+0000","last_undegraded":"2024-09-10T19:01:03.582877+0000","last_fullsized":"2024-09-10T19:01:03.582877+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_clean_scrub_stamp":"2024-09-10T19:00:56.075080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:05:20.475946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"60'19","reported_seq":47,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.582052+0000","last_change":"2024-09-10T19:00:53.524380+0000","last_active":"2024-09-10T19:01:03.582052+0000","last_peered":"2024-09-10T19:01:03.582052+0000","last_clean":"2024-09-10T19:01:03.582052+0000","last_became_active":"2024-09-10T19:00:53.523660+0000","last_became_peered":"2024-09-10T19:00:53.523660+0000","last_unstale":"2024-09-10T19:01:03.582052+0000","last_undegraded":"2024-09-10T19:01:03.582052+0000","last_fullsized":"2024-09-10T19:01:03.582052+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_clean_scrub_stamp":"2024-09-10T19:00:52.048288+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:04:20.329259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.519117+0000","last_change":"2024-09-10T19:00:51.060917+0000","last_active":"2024-09-10T19:01:03.519117+0000","last_peered":"2024-09-10T19:01:03.519117+0000","last_clean":"2024-09-10T19:01:03.519117+0000","last_became_active":"2024-09-10T19:00:51.060413+0000","last_became_peered":"2024-09-10T19:00:51.060413+0000","last_unstale":"2024-09-10T19:01:03.519117+0000","last_undegraded":"2024-09-10T19:01:03.519117+0000","last_fullsized":"2024-09-10T19:01:03.519117+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_clean_scrub_stamp":"2024-09-10T19:00:50.042140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:28:39.206225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.518566+0000","last_change":"2024-09-10T19:00:55.086478+0000","last_active":"2024-09-10T19:01:03.518566+0000","last_peered":"2024-09-10T19:01:03.518566+0000","last_clean":"2024-09-10T19:01:03.518566+0000","last_became_active":"2024-09-10T19:00:55.086185+0000","last_became_peered":"2024-09-10T19:00:55.086185+0000","last_unstale":"2024-09-10T19:01:03.518566+0000","last_undegraded":"2024-09-10T19:01:03.518566+0000","last_fullsized":"2024-09-10T19:01:03.518566+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_clean_scrub_stamp":"2024-09-10T19:00:54.064181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:23.935271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.581448+0000","last_change":"2024-09-10T19:00:57.102559+0000","last_active":"2024-09-10T19:01:03.581448+0000","last_peered":"2024-09-10T19:01:03.581448+0000","last_clean":"2024-09-10T19:01:03.581448+0000","last_became_active":"2024-09-10T19:00:57.102271+0000","last_became_peered":"2024-09-10T19:00:57.102271+0000","last_unstale":"2024-09-10T19:01:03.581448+0000","last_undegraded":"2024-09-10T19:01:03.581448+0000","last_fullsized":"2024-09-10T19:01:03.581448+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_clean_scrub_stamp":"2024-09-10T19:00:56.075080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:05:26.236726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"60'9","reported_seq":32,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.517649+0000","last_change":"2024-09-10T19:00:53.070558+0000","last_active":"2024-09-10T19:01:03.517649+0000","last_peered":"2024-09-10T19:01:03.517649+0000","last_clean":"2024-09-10T19:01:03.517649+0000","last_became_active":"2024-09-10T19:00:53.070183+0000","last_became_peered":"2024-09-10T19:00:53.070183+0000","last_unstale":"2024-09-10T19:01:03.517649+0000","last_undegraded":"2024-09-10T19:01:03.517649+0000","last_fullsized":"2024-09-10T19:01:03.517649+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_clean_scrub_stamp":"2024-09-10T19:00:52.048288+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:19:44.165302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.582714+0000","last_change":"2024-09-10T19:00:51.057966+0000","last_active":"2024-09-10T19:01:03.582714+0000","last_peered":"2024-09-10T19:01:03.582714+0000","last_clean":"2024-09-10T19:01:03.582714+0000","last_became_active":"2024-09-10T19:00:51.057450+0000","last_became_peered":"2024-09-10T19:00:51.057450+0000","last_unstale":"2024-09-10T19:01:03.582714+0000","last_undegraded":"2024-09-10T19:01:03.582714+0000","last_fullsized":"2024-09-10T19:01:03.582714+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_clean_scrub_stamp":"2024-09-10T19:00:50.042140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:06:38.881726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.518408+0000","last_change":"2024-09-10T19:00:55.085077+0000","last_active":"2024-09-10T19:01:03.518408+0000","last_peered":"2024-09-10T19:01:03.518408+0000","last_clean":"2024-09-10T19:01:03.518408+0000","last_became_active":"2024-09-10T19:00:55.084767+0000","last_became_peered":"2024-09-10T19:00:55.084767+0000","last_unstale":"2024-09-10T19:01:03.518408+0000","last_undegraded":"2024-09-10T19:01:03.518408+0000","last_fullsized":"2024-09-10T19:01:03.518408+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_clean_scrub_stamp":"2024-09-10T19:00:54.064181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:49:40.743259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.517020+0000","last_change":"2024-09-10T19:00:57.096433+0000","last_active":"2024-09-10T19:01:03.517020+0000","last_peered":"2024-09-10T19:01:03.517020+0000","last_clean":"2024-09-10T19:01:03.517020+0000","last_became_active":"2024-09-10T19:00:57.096137+0000","last_became_peered":"2024-09-10T19:00:57.096137+0000","last_unstale":"2024-09-10T19:01:03.517020+0000","last_undegraded":"2024-09-10T19:01:03.517020+0000","last_fullsized":"2024-09-10T19:01:03.517020+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_clean_scrub_stamp":"2024-09-10T19:00:56.075080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:06:47.792855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"60'15","reported_seq":41,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.581040+0000","last_change":"2024-09-10T19:00:53.524280+0000","last_active":"2024-09-10T19:01:03.581040+0000","last_peered":"2024-09-10T19:01:03.581040+0000","last_clean":"2024-09-10T19:01:03.581040+0000","last_became_active":"2024-09-10T19:00:53.524000+0000","last_became_peered":"2024-09-10T19:00:53.524000+0000","last_unstale":"2024-09-10T19:01:03.581040+0000","last_undegraded":"2024-09-10T19:01:03.581040+0000","last_fullsized":"2024-09-10T19:01:03.581040+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_clean_scrub_stamp":"2024-09-10T19:00:52.048288+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:21:26.121823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"53'1","reported_seq":27,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.519142+0000","last_change":"2024-09-10T19:00:51.059582+0000","last_active":"2024-09-10T19:01:03.519142+0000","last_peered":"2024-09-10T19:01:03.519142+0000","last_clean":"2024-09-10T19:01:03.519142+0000","last_became_active":"2024-09-10T19:00:51.059305+0000","last_became_peered":"2024-09-10T19:00:51.059305+0000","last_unstale":"2024-09-10T19:01:03.519142+0000","last_undegraded":"2024-09-10T19:01:03.519142+0000","last_fullsized":"2024-09-10T19:01:03.519142+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_clean_scrub_stamp":"2024-09-10T19:00:50.042140+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:01:52.779207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"60'8","reported_seq":40,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T19:01:59.170773+0000","last_change":"2024-09-10T19:00:55.087210+0000","last_active":"2024-09-10T19:01:59.170773+0000","last_peered":"2024-09-10T19:01:59.170773+0000","last_clean":"2024-09-10T19:01:59.170773+0000","last_became_active":"2024-09-10T19:00:55.086941+0000","last_became_peered":"2024-09-10T19:00:55.086941+0000","last_unstale":"2024-09-10T19:01:59.170773+0000","last_undegraded":"2024-09-10T19:01:59.170773+0000","last_fullsized":"2024-09-10T19:01:59.170773+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_clean_scrub_stamp":"2024-09-10T19:00:54.064181+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:43:01.581998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.581429+0000","last_change":"2024-09-10T19:00:57.179191+0000","last_active":"2024-09-10T19:01:03.581429+0000","last_peered":"2024-09-10T19:01:03.581429+0000","last_clean":"2024-09-10T19:01:03.581429+0000","last_became_active":"2024-09-10T19:00:57.178513+0000","last_became_peered":"2024-09-10T19:00:57.178513+0000","last_unstale":"2024-09-10T19:01:03.581429+0000","last_undegraded":"2024-09-10T19:01:03.581429+0000","last_fullsized":"2024-09-10T19:01:03.581429+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_clean_scrub_stamp":"2024-09-10T19:00:56.075080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:27:47.999862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"60'12","reported_seq":39,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.517351+0000","last_change":"2024-09-10T19:00:53.073475+0000","last_active":"2024-09-10T19:01:03.517351+0000","last_peered":"2024-09-10T19:01:03.517351+0000","last_clean":"2024-09-10T19:01:03.517351+0000","last_became_active":"2024-09-10T19:00:53.073182+0000","last_became_peered":"2024-09-10T19:00:53.073182+0000","last_unstale":"2024-09-10T19:01:03.517351+0000","last_undegraded":"2024-09-10T19:01:03.517351+0000","last_fullsized":"2024-09-10T19:01:03.517351+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_clean_scrub_stamp":"2024-09-10T19:00:52.048288+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:22:31.186050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"53'2","reported_seq":35,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.517912+0000","last_change":"2024-09-10T19:00:51.063141+0000","last_active":"2024-09-10T19:01:03.517912+0000","last_peered":"2024-09-10T19:01:03.517912+0000","last_clean":"2024-09-10T19:01:03.517912+0000","last_became_active":"2024-09-10T19:00:51.062863+0000","last_became_peered":"2024-09-10T19:00:51.062863+0000","last_unstale":"2024-09-10T19:01:03.517912+0000","last_undegraded":"2024-09-10T19:01:03.517912+0000","last_fullsized":"2024-09-10T19:01:03.517912+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_clean_scrub_stamp":"2024-09-10T19:00:50.042140+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:39.896731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.518263+0000","last_change":"2024-09-10T19:00:55.088366+0000","last_active":"2024-09-10T19:01:03.518263+0000","last_peered":"2024-09-10T19:01:03.518263+0000","last_clean":"2024-09-10T19:01:03.518263+0000","last_became_active":"2024-09-10T19:00:55.088070+0000","last_became_peered":"2024-09-10T19:00:55.088070+0000","last_unstale":"2024-09-10T19:01:03.518263+0000","last_undegraded":"2024-09-10T19:01:03.518263+0000","last_fullsized":"2024-09-10T19:01:03.518263+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_clean_scrub_stamp":"2024-09-10T19:00:54.064181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:11:23.003295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.581739+0000","last_change":"2024-09-10T19:00:57.098466+0000","last_active":"2024-09-10T19:01:03.581739+0000","last_peered":"2024-09-10T19:01:03.581739+0000","last_clean":"2024-09-10T19:01:03.581739+0000","last_became_active":"2024-09-10T19:00:57.098171+0000","last_became_peered":"2024-09-10T19:00:57.098171+0000","last_unstale":"2024-09-10T19:01:03.581739+0000","last_undegraded":"2024-09-10T19:01:03.581739+0000","last_fullsized":"2024-09-10T19:01:03.581739+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_clean_scrub_stamp":"2024-09-10T19:00:56.075080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:19:57.195328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"60'9","reported_seq":32,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.517576+0000","last_change":"2024-09-10T19:00:53.523852+0000","last_active":"2024-09-10T19:01:03.517576+0000","last_peered":"2024-09-10T19:01:03.517576+0000","last_clean":"2024-09-10T19:01:03.517576+0000","last_became_active":"2024-09-10T19:00:53.523509+0000","last_became_peered":"2024-09-10T19:00:53.523509+0000","last_unstale":"2024-09-10T19:01:03.517576+0000","last_undegraded":"2024-09-10T19:01:03.517576+0000","last_fullsized":"2024-09-10T19:01:03.517576+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_clean_scrub_stamp":"2024-09-10T19:00:52.048288+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:29:11.861725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.518341+0000","last_change":"2024-09-10T19:00:51.061438+0000","last_active":"2024-09-10T19:01:03.518341+0000","last_peered":"2024-09-10T19:01:03.518341+0000","last_clean":"2024-09-10T19:01:03.518341+0000","last_became_active":"2024-09-10T19:00:51.061174+0000","last_became_peered":"2024-09-10T19:00:51.061174+0000","last_unstale":"2024-09-10T19:01:03.518341+0000","last_undegraded":"2024-09-10T19:01:03.518341+0000","last_fullsized":"2024-09-10T19:01:03.518341+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_clean_scrub_stamp":"2024-09-10T19:00:50.042140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:25:41.385708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.582575+0000","last_change":"2024-09-10T19:00:55.090438+0000","last_active":"2024-09-10T19:01:03.582575+0000","last_peered":"2024-09-10T19:01:03.582575+0000","last_clean":"2024-09-10T19:01:03.582575+0000","last_became_active":"2024-09-10T19:00:55.090102+0000","last_became_peered":"2024-09-10T19:00:55.090102+0000","last_unstale":"2024-09-10T19:01:03.582575+0000","last_undegraded":"2024-09-10T19:01:03.582575+0000","last_fullsized":"2024-09-10T19:01:03.582575+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_clean_scrub_stamp":"2024-09-10T19:00:54.064181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:32:09.171597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.518642+0000","last_change":"2024-09-10T19:00:57.096878+0000","last_active":"2024-09-10T19:01:03.518642+0000","last_peered":"2024-09-10T19:01:03.518642+0000","last_clean":"2024-09-10T19:01:03.518642+0000","last_became_active":"2024-09-10T19:00:57.096566+0000","last_became_peered":"2024-09-10T19:00:57.096566+0000","last_unstale":"2024-09-10T19:01:03.518642+0000","last_undegraded":"2024-09-10T19:01:03.518642+0000","last_fullsized":"2024-09-10T19:01:03.518642+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_clean_scrub_stamp":"2024-09-10T19:00:56.075080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:30:52.162107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.581569+0000","last_change":"2024-09-10T19:00:51.056880+0000","last_active":"2024-09-10T19:01:03.581569+0000","last_peered":"2024-09-10T19:01:03.581569+0000","last_clean":"2024-09-10T19:01:03.581569+0000","last_became_active":"2024-09-10T19:00:51.056586+0000","last_became_peered":"2024-09-10T19:00:51.056586+0000","last_unstale":"2024-09-10T19:01:03.581569+0000","last_undegraded":"2024-09-10T19:01:03.581569+0000","last_fullsized":"2024-09-10T19:01:03.581569+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_clean_scrub_stamp":"2024-09-10T19:00:50.042140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:14.070058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"60'6","reported_seq":25,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.582714+0000","last_change":"2024-09-10T19:00:53.524929+0000","last_active":"2024-09-10T19:01:03.582714+0000","last_peered":"2024-09-10T19:01:03.582714+0000","last_clean":"2024-09-10T19:01:03.582714+0000","last_became_active":"2024-09-10T19:00:53.524608+0000","last_became_peered":"2024-09-10T19:00:53.524608+0000","last_unstale":"2024-09-10T19:01:03.582714+0000","last_undegraded":"2024-09-10T19:01:03.582714+0000","last_fullsized":"2024-09-10T19:01:03.582714+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_clean_scrub_stamp":"2024-09-10T19:00:52.048288+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:32.528418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.582494+0000","last_change":"2024-09-10T19:00:55.094110+0000","last_active":"2024-09-10T19:01:03.582494+0000","last_peered":"2024-09-10T19:01:03.582494+0000","last_clean":"2024-09-10T19:01:03.582494+0000","last_became_active":"2024-09-10T19:00:55.093711+0000","last_became_peered":"2024-09-10T19:00:55.093711+0000","last_unstale":"2024-09-10T19:01:03.582494+0000","last_undegraded":"2024-09-10T19:01:03.582494+0000","last_fullsized":"2024-09-10T19:01:03.582494+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_clean_scrub_stamp":"2024-09-10T19:00:54.064181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:06.254108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.582009+0000","last_change":"2024-09-10T19:00:57.179506+0000","last_active":"2024-09-10T19:01:03.582009+0000","last_peered":"2024-09-10T19:01:03.582009+0000","last_clean":"2024-09-10T19:01:03.582009+0000","last_became_active":"2024-09-10T19:00:57.179095+0000","last_became_peered":"2024-09-10T19:00:57.179095+0000","last_unstale":"2024-09-10T19:01:03.582009+0000","last_undegraded":"2024-09-10T19:01:03.582009+0000","last_fullsized":"2024-09-10T19:01:03.582009+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_clean_scrub_stamp":"2024-09-10T19:00:56.075080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:48.626244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"60'10","reported_seq":31,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.582452+0000","last_change":"2024-09-10T19:00:53.525248+0000","last_active":"2024-09-10T19:01:03.582452+0000","last_peered":"2024-09-10T19:01:03.582452+0000","last_clean":"2024-09-10T19:01:03.582452+0000","last_became_active":"2024-09-10T19:00:53.524814+0000","last_became_peered":"2024-09-10T19:00:53.524814+0000","last_unstale":"2024-09-10T19:01:03.582452+0000","last_undegraded":"2024-09-10T19:01:03.582452+0000","last_fullsized":"2024-09-10T19:01:03.582452+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_clean_scrub_stamp":"2024-09-10T19:00:52.048288+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:04.038219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.581606+0000","last_change":"2024-09-10T19:00:51.064379+0000","last_active":"2024-09-10T19:01:03.581606+0000","last_peered":"2024-09-10T19:01:03.581606+0000","last_clean":"2024-09-10T19:01:03.581606+0000","last_became_active":"2024-09-10T19:00:51.064125+0000","last_became_peered":"2024-09-10T19:00:51.064125+0000","last_unstale":"2024-09-10T19:01:03.581606+0000","last_undegraded":"2024-09-10T19:01:03.581606+0000","last_fullsized":"2024-09-10T19:01:03.581606+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_clean_scrub_stamp":"2024-09-10T19:00:50.042140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:02:29.351420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"60'8","reported_seq":37,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T19:01:59.170778+0000","last_change":"2024-09-10T19:00:55.094960+0000","last_active":"2024-09-10T19:01:59.170778+0000","last_peered":"2024-09-10T19:01:59.170778+0000","last_clean":"2024-09-10T19:01:59.170778+0000","last_became_active":"2024-09-10T19:00:55.094474+0000","last_became_peered":"2024-09-10T19:00:55.094474+0000","last_unstale":"2024-09-10T19:01:59.170778+0000","last_undegraded":"2024-09-10T19:01:59.170778+0000","last_fullsized":"2024-09-10T19:01:59.170778+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_clean_scrub_stamp":"2024-09-10T19:00:54.064181+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:09.759680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.517202+0000","last_change":"2024-09-10T19:00:57.095841+0000","last_active":"2024-09-10T19:01:03.517202+0000","last_peered":"2024-09-10T19:01:03.517202+0000","last_clean":"2024-09-10T19:01:03.517202+0000","last_became_active":"2024-09-10T19:00:57.095540+0000","last_became_peered":"2024-09-10T19:00:57.095540+0000","last_unstale":"2024-09-10T19:01:03.517202+0000","last_undegraded":"2024-09-10T19:01:03.517202+0000","last_fullsized":"2024-09-10T19:01:03.517202+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_clean_scrub_stamp":"2024-09-10T19:00:56.075080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:16:13.910364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"60'9","reported_seq":32,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.582924+0000","last_change":"2024-09-10T19:00:53.525632+0000","last_active":"2024-09-10T19:01:03.582924+0000","last_peered":"2024-09-10T19:01:03.582924+0000","last_clean":"2024-09-10T19:01:03.582924+0000","last_became_active":"2024-09-10T19:00:53.525306+0000","last_became_peered":"2024-09-10T19:00:53.525306+0000","last_unstale":"2024-09-10T19:01:03.582924+0000","last_undegraded":"2024-09-10T19:01:03.582924+0000","last_fullsized":"2024-09-10T19:01:03.582924+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_clean_scrub_stamp":"2024-09-10T19:00:52.048288+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:52:17.846341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.517816+0000","last_change":"2024-09-10T19:00:51.058623+0000","last_active":"2024-09-10T19:01:03.517816+0000","last_peered":"2024-09-10T19:01:03.517816+0000","last_clean":"2024-09-10T19:01:03.517816+0000","last_became_active":"2024-09-10T19:00:51.058331+0000","last_became_peered":"2024-09-10T19:00:51.058331+0000","last_unstale":"2024-09-10T19:01:03.517816+0000","last_undegraded":"2024-09-10T19:01:03.517816+0000","last_fullsized":"2024-09-10T19:01:03.517816+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_clean_scrub_stamp":"2024-09-10T19:00:50.042140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:50:02.471740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"60'8","reported_seq":37,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T19:01:59.170766+0000","last_change":"2024-09-10T19:00:55.082961+0000","last_active":"2024-09-10T19:01:59.170766+0000","last_peered":"2024-09-10T19:01:59.170766+0000","last_clean":"2024-09-10T19:01:59.170766+0000","last_became_active":"2024-09-10T19:00:55.082689+0000","last_became_peered":"2024-09-10T19:00:55.082689+0000","last_unstale":"2024-09-10T19:01:59.170766+0000","last_undegraded":"2024-09-10T19:01:59.170766+0000","last_fullsized":"2024-09-10T19:01:59.170766+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_clean_scrub_stamp":"2024-09-10T19:00:54.064181+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:43:19.270561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.517678+0000","last_change":"2024-09-10T19:00:57.095311+0000","last_active":"2024-09-10T19:01:03.517678+0000","last_peered":"2024-09-10T19:01:03.517678+0000","last_clean":"2024-09-10T19:01:03.517678+0000","last_became_active":"2024-09-10T19:00:57.095028+0000","last_became_peered":"2024-09-10T19:00:57.095028+0000","last_unstale":"2024-09-10T19:01:03.517678+0000","last_undegraded":"2024-09-10T19:01:03.517678+0000","last_fullsized":"2024-09-10T19:01:03.517678+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_clean_scrub_stamp":"2024-09-10T19:00:56.075080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:19:56.698879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"60'9","reported_seq":32,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.581681+0000","last_change":"2024-09-10T19:00:53.074967+0000","last_active":"2024-09-10T19:01:03.581681+0000","last_peered":"2024-09-10T19:01:03.581681+0000","last_clean":"2024-09-10T19:01:03.581681+0000","last_became_active":"2024-09-10T19:00:53.073099+0000","last_became_peered":"2024-09-10T19:00:53.073099+0000","last_unstale":"2024-09-10T19:01:03.581681+0000","last_undegraded":"2024-09-10T19:01:03.581681+0000","last_fullsized":"2024-09-10T19:01:03.581681+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_clean_scrub_stamp":"2024-09-10T19:00:52.048288+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:44:13.223102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.581970+0000","last_change":"2024-09-10T19:00:51.065518+0000","last_active":"2024-09-10T19:01:03.581970+0000","last_peered":"2024-09-10T19:01:03.581970+0000","last_clean":"2024-09-10T19:01:03.581970+0000","last_became_active":"2024-09-10T19:00:51.065174+0000","last_became_peered":"2024-09-10T19:00:51.065174+0000","last_unstale":"2024-09-10T19:01:03.581970+0000","last_undegraded":"2024-09-10T19:01:03.581970+0000","last_fullsized":"2024-09-10T19:01:03.581970+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_clean_scrub_stamp":"2024-09-10T19:00:50.042140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:52:06.268154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.581825+0000","last_change":"2024-09-10T19:00:55.091574+0000","last_active":"2024-09-10T19:01:03.581825+0000","last_peered":"2024-09-10T19:01:03.581825+0000","last_clean":"2024-09-10T19:01:03.581825+0000","last_became_active":"2024-09-10T19:00:55.090983+0000","last_became_peered":"2024-09-10T19:00:55.090983+0000","last_unstale":"2024-09-10T19:01:03.581825+0000","last_undegraded":"2024-09-10T19:01:03.581825+0000","last_fullsized":"2024-09-10T19:01:03.581825+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_clean_scrub_stamp":"2024-09-10T19:00:54.064181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:38:25.954120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.516810+0000","last_change":"2024-09-10T19:00:57.098571+0000","last_active":"2024-09-10T19:01:03.516810+0000","last_peered":"2024-09-10T19:01:03.516810+0000","last_clean":"2024-09-10T19:01:03.516810+0000","last_became_active":"2024-09-10T19:00:57.097938+0000","last_became_peered":"2024-09-10T19:00:57.097938+0000","last_unstale":"2024-09-10T19:01:03.516810+0000","last_undegraded":"2024-09-10T19:01:03.516810+0000","last_fullsized":"2024-09-10T19:01:03.516810+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_clean_scrub_stamp":"2024-09-10T19:00:56.075080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:19:14.768601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"60'11","reported_seq":35,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.517368+0000","last_change":"2024-09-10T19:00:53.065637+0000","last_active":"2024-09-10T19:01:03.517368+0000","last_peered":"2024-09-10T19:01:03.517368+0000","last_clean":"2024-09-10T19:01:03.517368+0000","last_became_active":"2024-09-10T19:00:53.065288+0000","last_became_peered":"2024-09-10T19:00:53.065288+0000","last_unstale":"2024-09-10T19:01:03.517368+0000","last_undegraded":"2024-09-10T19:01:03.517368+0000","last_fullsized":"2024-09-10T19:01:03.517368+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_clean_scrub_stamp":"2024-09-10T19:00:52.048288+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:00:39.835523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.582374+0000","last_change":"2024-09-10T19:00:51.063187+0000","last_active":"2024-09-10T19:01:03.582374+0000","last_peered":"2024-09-10T19:01:03.582374+0000","last_clean":"2024-09-10T19:01:03.582374+0000","last_became_active":"2024-09-10T19:00:51.062701+0000","last_became_peered":"2024-09-10T19:00:51.062701+0000","last_unstale":"2024-09-10T19:01:03.582374+0000","last_undegraded":"2024-09-10T19:01:03.582374+0000","last_fullsized":"2024-09-10T19:01:03.582374+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_clean_scrub_stamp":"2024-09-10T19:00:50.042140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:04:11.575350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.581506+0000","last_change":"2024-09-10T19:00:55.093170+0000","last_active":"2024-09-10T19:01:03.581506+0000","last_peered":"2024-09-10T19:01:03.581506+0000","last_clean":"2024-09-10T19:01:03.581506+0000","last_became_active":"2024-09-10T19:00:55.092469+0000","last_became_peered":"2024-09-10T19:00:55.092469+0000","last_unstale":"2024-09-10T19:01:03.581506+0000","last_undegraded":"2024-09-10T19:01:03.581506+0000","last_fullsized":"2024-09-10T19:01:03.581506+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_clean_scrub_stamp":"2024-09-10T19:00:54.064181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:58:11.788948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.581562+0000","last_change":"2024-09-10T19:00:57.096682+0000","last_active":"2024-09-10T19:01:03.581562+0000","last_peered":"2024-09-10T19:01:03.581562+0000","last_clean":"2024-09-10T19:01:03.581562+0000","last_became_active":"2024-09-10T19:00:57.096393+0000","last_became_peered":"2024-09-10T19:00:57.096393+0000","last_unstale":"2024-09-10T19:01:03.581562+0000","last_undegraded":"2024-09-10T19:01:03.581562+0000","last_fullsized":"2024-09-10T19:01:03.581562+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_clean_scrub_stamp":"2024-09-10T19:00:56.075080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:58:52.734339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"60'4","reported_seq":22,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.581169+0000","last_change":"2024-09-10T19:00:53.071584+0000","last_active":"2024-09-10T19:01:03.581169+0000","last_peered":"2024-09-10T19:01:03.581169+0000","last_clean":"2024-09-10T19:01:03.581169+0000","last_became_active":"2024-09-10T19:00:53.070210+0000","last_became_peered":"2024-09-10T19:00:53.070210+0000","last_unstale":"2024-09-10T19:01:03.581169+0000","last_undegraded":"2024-09-10T19:01:03.581169+0000","last_fullsized":"2024-09-10T19:01:03.581169+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_clean_scrub_stamp":"2024-09-10T19:00:52.048288+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:55:11.894284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.582648+0000","last_change":"2024-09-10T19:00:51.064601+0000","last_active":"2024-09-10T19:01:03.582648+0000","last_peered":"2024-09-10T19:01:03.582648+0000","last_clean":"2024-09-10T19:01:03.582648+0000","last_became_active":"2024-09-10T19:00:51.064331+0000","last_became_peered":"2024-09-10T19:00:51.064331+0000","last_unstale":"2024-09-10T19:01:03.582648+0000","last_undegraded":"2024-09-10T19:01:03.582648+0000","last_fullsized":"2024-09-10T19:01:03.582648+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_clean_scrub_stamp":"2024-09-10T19:00:50.042140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:06:48.443784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.581340+0000","last_change":"2024-09-10T19:00:55.089871+0000","last_active":"2024-09-10T19:01:03.581340+0000","last_peered":"2024-09-10T19:01:03.581340+0000","last_clean":"2024-09-10T19:01:03.581340+0000","last_became_active":"2024-09-10T19:00:55.089614+0000","last_became_peered":"2024-09-10T19:00:55.089614+0000","last_unstale":"2024-09-10T19:01:03.581340+0000","last_undegraded":"2024-09-10T19:01:03.581340+0000","last_fullsized":"2024-09-10T19:01:03.581340+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_clean_scrub_stamp":"2024-09-10T19:00:54.064181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:23:07.238782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.581159+0000","last_change":"2024-09-10T19:00:57.096479+0000","last_active":"2024-09-10T19:01:03.581159+0000","last_peered":"2024-09-10T19:01:03.581159+0000","last_clean":"2024-09-10T19:01:03.581159+0000","last_became_active":"2024-09-10T19:00:57.096194+0000","last_became_peered":"2024-09-10T19:00:57.096194+0000","last_unstale":"2024-09-10T19:01:03.581159+0000","last_undegraded":"2024-09-10T19:01:03.581159+0000","last_fullsized":"2024-09-10T19:01:03.581159+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_clean_scrub_stamp":"2024-09-10T19:00:56.075080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:26:20.434998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"60'11","reported_seq":35,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.581286+0000","last_change":"2024-09-10T19:00:53.523667+0000","last_active":"2024-09-10T19:01:03.581286+0000","last_peered":"2024-09-10T19:01:03.581286+0000","last_clean":"2024-09-10T19:01:03.581286+0000","last_became_active":"2024-09-10T19:00:53.523357+0000","last_became_peered":"2024-09-10T19:00:53.523357+0000","last_unstale":"2024-09-10T19:01:03.581286+0000","last_undegraded":"2024-09-10T19:01:03.581286+0000","last_fullsized":"2024-09-10T19:01:03.581286+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_clean_scrub_stamp":"2024-09-10T19:00:52.048288+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:16:44.928708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.582147+0000","last_change":"2024-09-10T19:00:51.061211+0000","last_active":"2024-09-10T19:01:03.582147+0000","last_peered":"2024-09-10T19:01:03.582147+0000","last_clean":"2024-09-10T19:01:03.582147+0000","last_became_active":"2024-09-10T19:00:51.060084+0000","last_became_peered":"2024-09-10T19:00:51.060084+0000","last_unstale":"2024-09-10T19:01:03.582147+0000","last_undegraded":"2024-09-10T19:01:03.582147+0000","last_fullsized":"2024-09-10T19:01:03.582147+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_clean_scrub_stamp":"2024-09-10T19:00:50.042140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:26.265240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.581026+0000","last_change":"2024-09-10T19:00:55.087781+0000","last_active":"2024-09-10T19:01:03.581026+0000","last_peered":"2024-09-10T19:01:03.581026+0000","last_clean":"2024-09-10T19:01:03.581026+0000","last_became_active":"2024-09-10T19:00:55.087396+0000","last_became_peered":"2024-09-10T19:00:55.087396+0000","last_unstale":"2024-09-10T19:01:03.581026+0000","last_undegraded":"2024-09-10T19:01:03.581026+0000","last_fullsized":"2024-09-10T19:01:03.581026+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_clean_scrub_stamp":"2024-09-10T19:00:54.064181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:00:02.924581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.581656+0000","last_change":"2024-09-10T19:00:57.179547+0000","last_active":"2024-09-10T19:01:03.581656+0000","last_peered":"2024-09-10T19:01:03.581656+0000","last_clean":"2024-09-10T19:01:03.581656+0000","last_became_active":"2024-09-10T19:00:57.178628+0000","last_became_peered":"2024-09-10T19:00:57.178628+0000","last_unstale":"2024-09-10T19:01:03.581656+0000","last_undegraded":"2024-09-10T19:01:03.581656+0000","last_fullsized":"2024-09-10T19:01:03.581656+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_clean_scrub_stamp":"2024-09-10T19:00:56.075080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:50:12.142176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"60'1","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.582192+0000","last_change":"2024-09-10T19:00:57.099261+0000","last_active":"2024-09-10T19:01:03.582192+0000","last_peered":"2024-09-10T19:01:03.582192+0000","last_clean":"2024-09-10T19:01:03.582192+0000","last_became_active":"2024-09-10T19:00:57.098986+0000","last_became_peered":"2024-09-10T19:00:57.098986+0000","last_unstale":"2024-09-10T19:01:03.582192+0000","last_undegraded":"2024-09-10T19:01:03.582192+0000","last_fullsized":"2024-09-10T19:01:03.582192+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_clean_scrub_stamp":"2024-09-10T19:00:56.075080+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:19:39.659567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.581062+0000","last_change":"2024-09-10T19:00:51.068124+0000","last_active":"2024-09-10T19:01:03.581062+0000","last_peered":"2024-09-10T19:01:03.581062+0000","last_clean":"2024-09-10T19:01:03.581062+0000","last_became_active":"2024-09-10T19:00:51.067818+0000","last_became_peered":"2024-09-10T19:00:51.067818+0000","last_unstale":"2024-09-10T19:01:03.581062+0000","last_undegraded":"2024-09-10T19:01:03.581062+0000","last_fullsized":"2024-09-10T19:01:03.581062+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_clean_scrub_stamp":"2024-09-10T19:00:50.042140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:38:41.252963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"60'10","reported_seq":31,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.517483+0000","last_change":"2024-09-10T19:00:53.526661+0000","last_active":"2024-09-10T19:01:03.517483+0000","last_peered":"2024-09-10T19:01:03.517483+0000","last_clean":"2024-09-10T19:01:03.517483+0000","last_became_active":"2024-09-10T19:00:53.526412+0000","last_became_peered":"2024-09-10T19:00:53.526412+0000","last_unstale":"2024-09-10T19:01:03.517483+0000","last_undegraded":"2024-09-10T19:01:03.517483+0000","last_fullsized":"2024-09-10T19:01:03.517483+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_clean_scrub_stamp":"2024-09-10T19:00:52.048288+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:00:46.966519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"60'8","reported_seq":40,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T19:01:59.170810+0000","last_change":"2024-09-10T19:00:55.091309+0000","last_active":"2024-09-10T19:01:59.170810+0000","last_peered":"2024-09-10T19:01:59.170810+0000","last_clean":"2024-09-10T19:01:59.170810+0000","last_became_active":"2024-09-10T19:00:55.090505+0000","last_became_peered":"2024-09-10T19:00:55.090505+0000","last_unstale":"2024-09-10T19:01:59.170810+0000","last_undegraded":"2024-09-10T19:01:59.170810+0000","last_fullsized":"2024-09-10T19:01:59.170810+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_clean_scrub_stamp":"2024-09-10T19:00:54.064181+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:56:15.209664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.581761+0000","last_change":"2024-09-10T19:00:57.098698+0000","last_active":"2024-09-10T19:01:03.581761+0000","last_peered":"2024-09-10T19:01:03.581761+0000","last_clean":"2024-09-10T19:01:03.581761+0000","last_became_active":"2024-09-10T19:00:57.098407+0000","last_became_peered":"2024-09-10T19:00:57.098407+0000","last_unstale":"2024-09-10T19:01:03.581761+0000","last_undegraded":"2024-09-10T19:01:03.581761+0000","last_fullsized":"2024-09-10T19:01:03.581761+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_clean_scrub_stamp":"2024-09-10T19:00:56.075080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:12:08.606980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.581450+0000","last_change":"2024-09-10T19:00:51.058555+0000","last_active":"2024-09-10T19:01:03.581450+0000","last_peered":"2024-09-10T19:01:03.581450+0000","last_clean":"2024-09-10T19:01:03.581450+0000","last_became_active":"2024-09-10T19:00:51.058245+0000","last_became_peered":"2024-09-10T19:00:51.058245+0000","last_unstale":"2024-09-10T19:01:03.581450+0000","last_undegraded":"2024-09-10T19:01:03.581450+0000","last_fullsized":"2024-09-10T19:01:03.581450+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_clean_scrub_stamp":"2024-09-10T19:00:50.042140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:57.167503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"60'11","reported_seq":35,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.582070+0000","last_change":"2024-09-10T19:00:53.068178+0000","last_active":"2024-09-10T19:01:03.582070+0000","last_peered":"2024-09-10T19:01:03.582070+0000","last_clean":"2024-09-10T19:01:03.582070+0000","last_became_active":"2024-09-10T19:00:53.067863+0000","last_became_peered":"2024-09-10T19:00:53.067863+0000","last_unstale":"2024-09-10T19:01:03.582070+0000","last_undegraded":"2024-09-10T19:01:03.582070+0000","last_fullsized":"2024-09-10T19:01:03.582070+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_clean_scrub_stamp":"2024-09-10T19:00:52.048288+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:57.926892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.517372+0000","last_change":"2024-09-10T19:00:55.086482+0000","last_active":"2024-09-10T19:01:03.517372+0000","last_peered":"2024-09-10T19:01:03.517372+0000","last_clean":"2024-09-10T19:01:03.517372+0000","last_became_active":"2024-09-10T19:00:55.086198+0000","last_became_peered":"2024-09-10T19:00:55.086198+0000","last_unstale":"2024-09-10T19:01:03.517372+0000","last_undegraded":"2024-09-10T19:01:03.517372+0000","last_fullsized":"2024-09-10T19:01:03.517372+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_clean_scrub_stamp":"2024-09-10T19:00:54.064181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:41:24.304338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.517625+0000","last_change":"2024-09-10T19:00:57.178470+0000","last_active":"2024-09-10T19:01:03.517625+0000","last_peered":"2024-09-10T19:01:03.517625+0000","last_clean":"2024-09-10T19:01:03.517625+0000","last_became_active":"2024-09-10T19:00:57.178092+0000","last_became_peered":"2024-09-10T19:00:57.178092+0000","last_unstale":"2024-09-10T19:01:03.517625+0000","last_undegraded":"2024-09-10T19:01:03.517625+0000","last_fullsized":"2024-09-10T19:01:03.517625+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_clean_scrub_stamp":"2024-09-10T19:00:56.075080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:32:40.554756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"53'1","reported_seq":25,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.582184+0000","last_change":"2024-09-10T19:00:51.063736+0000","last_active":"2024-09-10T19:01:03.582184+0000","last_peered":"2024-09-10T19:01:03.582184+0000","last_clean":"2024-09-10T19:01:03.582184+0000","last_became_active":"2024-09-10T19:00:51.060346+0000","last_became_peered":"2024-09-10T19:00:51.060346+0000","last_unstale":"2024-09-10T19:01:03.582184+0000","last_undegraded":"2024-09-10T19:01:03.582184+0000","last_fullsized":"2024-09-10T19:01:03.582184+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_clean_scrub_stamp":"2024-09-10T19:00:50.042140+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:01:10.085465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"60'15","reported_seq":41,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.518861+0000","last_change":"2024-09-10T19:00:53.066353+0000","last_active":"2024-09-10T19:01:03.518861+0000","last_peered":"2024-09-10T19:01:03.518861+0000","last_clean":"2024-09-10T19:01:03.518861+0000","last_became_active":"2024-09-10T19:00:53.065958+0000","last_became_peered":"2024-09-10T19:00:53.065958+0000","last_unstale":"2024-09-10T19:01:03.518861+0000","last_undegraded":"2024-09-10T19:01:03.518861+0000","last_fullsized":"2024-09-10T19:01:03.518861+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_clean_scrub_stamp":"2024-09-10T19:00:52.048288+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:42:46.537217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.582195+0000","last_change":"2024-09-10T19:00:55.093214+0000","last_active":"2024-09-10T19:01:03.582195+0000","last_peered":"2024-09-10T19:01:03.582195+0000","last_clean":"2024-09-10T19:01:03.582195+0000","last_became_active":"2024-09-10T19:00:55.092890+0000","last_became_peered":"2024-09-10T19:00:55.092890+0000","last_unstale":"2024-09-10T19:01:03.582195+0000","last_undegraded":"2024-09-10T19:01:03.582195+0000","last_fullsized":"2024-09-10T19:01:03.582195+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_clean_scrub_stamp":"2024-09-10T19:00:54.064181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:28:15.101444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.582491+0000","last_change":"2024-09-10T19:00:57.178506+0000","last_active":"2024-09-10T19:01:03.582491+0000","last_peered":"2024-09-10T19:01:03.582491+0000","last_clean":"2024-09-10T19:01:03.582491+0000","last_became_active":"2024-09-10T19:00:57.178181+0000","last_became_peered":"2024-09-10T19:00:57.178181+0000","last_unstale":"2024-09-10T19:01:03.582491+0000","last_undegraded":"2024-09-10T19:01:03.582491+0000","last_fullsized":"2024-09-10T19:01:03.582491+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_clean_scrub_stamp":"2024-09-10T19:00:56.075080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:49.394838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.582069+0000","last_change":"2024-09-10T19:00:51.061326+0000","last_active":"2024-09-10T19:01:03.582069+0000","last_peered":"2024-09-10T19:01:03.582069+0000","last_clean":"2024-09-10T19:01:03.582069+0000","last_became_active":"2024-09-10T19:00:51.059882+0000","last_became_peered":"2024-09-10T19:00:51.059882+0000","last_unstale":"2024-09-10T19:01:03.582069+0000","last_undegraded":"2024-09-10T19:01:03.582069+0000","last_fullsized":"2024-09-10T19:01:03.582069+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_clean_scrub_stamp":"2024-09-10T19:00:50.042140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:05:04.740122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"60'12","reported_seq":39,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.582180+0000","last_change":"2024-09-10T19:00:53.073818+0000","last_active":"2024-09-10T19:01:03.582180+0000","last_peered":"2024-09-10T19:01:03.582180+0000","last_clean":"2024-09-10T19:01:03.582180+0000","last_became_active":"2024-09-10T19:00:53.073370+0000","last_became_peered":"2024-09-10T19:00:53.073370+0000","last_unstale":"2024-09-10T19:01:03.582180+0000","last_undegraded":"2024-09-10T19:01:03.582180+0000","last_fullsized":"2024-09-10T19:01:03.582180+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_clean_scrub_stamp":"2024-09-10T19:00:52.048288+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:48:31.098509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.517498+0000","last_change":"2024-09-10T19:00:55.092447+0000","last_active":"2024-09-10T19:01:03.517498+0000","last_peered":"2024-09-10T19:01:03.517498+0000","last_clean":"2024-09-10T19:01:03.517498+0000","last_became_active":"2024-09-10T19:00:55.092155+0000","last_became_peered":"2024-09-10T19:00:55.092155+0000","last_unstale":"2024-09-10T19:01:03.517498+0000","last_undegraded":"2024-09-10T19:01:03.517498+0000","last_fullsized":"2024-09-10T19:01:03.517498+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_clean_scrub_stamp":"2024-09-10T19:00:54.064181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:45:52.505237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":391,"ondisk_log_size":391,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":70,"num_read_kb":65,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":49,"seq":210453397524,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298552,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446600,"statfs":{"total":95995035648,"available":95689318400,"internally_reserved":0,"allocated":1224704,"data_stored":826192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 19:01:39 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.621,"5min":0.621,"15min":0.621},"last":0.61899999999999999}]},{"osd":1,"last update":"Tue Sep 10 19:01:39 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.64400000000000002}]},{"osd":2,"last update":"Tue Sep 10 19:01:39 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.60499999999999998}]},{"osd":3,"last update":"Tue Sep 10 19:01:39 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.55200000000000005}]},{"osd":4,"last update":"Tue Sep 10 19:01:39 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.439,"5min":0.439,"15min":0.439},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.439}]},{"osd":5,"last update":"Tue Sep 10 19:01:39 2024","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.379},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.40500000000000003}]},{"osd":6,"last update":"Tue Sep 10 19:01:39 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.48099999999999998}]}]},{"osd":6,"up_from":43,"seq":184683593752,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298524,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446628,"statfs":{"total":95995035648,"available":95689347072,"internally_reserved":0,"allocated":1200128,"data_stored":824717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 19:01:19 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.9399999999999999,"5min":1.9399999999999999,"15min":1.9399999999999999},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":2.484,"5min":2.484,"15min":2.484},"last":0.51500000000000001}]},{"osd":1,"last update":"Tue Sep 10 19:01:19 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":2.9569999999999999,"5min":2.9569999999999999,"15min":2.9569999999999999},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.8859999999999999,"5min":1.8859999999999999,"15min":1.8859999999999999},"last":0.46899999999999997}]},{"osd":2,"last update":"Tue Sep 10 19:01:19 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.9119999999999999,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":0.498},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":2.5169999999999999,"5min":2.5169999999999999,"15min":2.5169999999999999},"last":0.44500000000000001}]},{"osd":3,"last update":"Tue Sep 10 19:01:19 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":2.9910000000000001,"5min":2.9910000000000001,"15min":2.9910000000000001},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.8520000000000001,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":0.53600000000000003}]},{"osd":4,"last update":"Tue Sep 10 19:01:19 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":3.484,"5min":3.484,"15min":3.484},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":3.9300000000000002,"5min":3.9300000000000002,"15min":3.9300000000000002},"last":0.34699999999999998}]},{"osd":5,"last update":"Tue Sep 10 19:01:19 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":3.2320000000000002,"5min":3.2320000000000002,"15min":3.2320000000000002},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":3.331,"5min":3.331,"15min":3.331},"last":0.38600000000000001}]},{"osd":7,"last update":"Tue Sep 10 19:01:41 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":4.1680000000000001,"5min":4.1680000000000001,"15min":4.1680000000000001},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":4.2000000000000002,"5min":4.2000000000000002,"15min":4.2000000000000002},"last":0.58299999999999996}]}]},{"osd":5,"up_from":37,"seq":158913789980,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297940,"kb_used_data":584,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447212,"statfs":{"total":95995035648,"available":95689945088,"internally_reserved":0,"allocated":598016,"data_stored":233608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 19:01:54 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.51200000000000001,"15min":0.50800000000000001},"min":{"1min":0.40999999999999998,"5min":0.312,"15min":0.312},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.47799999999999998,"15min":0.47599999999999998},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.54300000000000004}]},{"osd":1,"last update":"Tue Sep 10 19:01:54 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.48899999999999999,"15min":0.48399999999999999},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.50600000000000001,"15min":0.504},"min":{"1min":0.42399999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.67200000000000004,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.50600000000000001}]},{"osd":2,"last update":"Tue Sep 10 19:01:54 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.501,"15min":0.501},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.36199999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.69799999999999995,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.47399999999999998}]},{"osd":3,"last update":"Tue Sep 10 19:01:54 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.38900000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.59799999999999998}]},{"osd":4,"last update":"Tue Sep 10 19:01:54 2024","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.377,"15min":0.376},"min":{"1min":0.308,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.47699999999999998,"5min":0.629,"15min":0.629},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.372,"15min":0.36799999999999999},"min":{"1min":0.32100000000000001,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.499,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.61299999999999999}]},{"osd":6,"last update":"Tue Sep 10 19:01:18 2024","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.52700000000000002}]},{"osd":7,"last update":"Tue Sep 10 19:01:43 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.63100000000000001}]}]},{"osd":4,"up_from":31,"seq":133143986210,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297972,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447180,"statfs":{"total":95995035648,"available":95689912320,"internally_reserved":0,"allocated":630784,"data_stored":234801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 19:01:31 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.32100000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.49199999999999999,"15min":0.49399999999999999},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.69399999999999995,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.48099999999999998}]},{"osd":1,"last update":"Tue Sep 10 19:01:31 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48999999999999999,"15min":0.49099999999999999},"min":{"1min":0.36399999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.504,"15min":0.5},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.39100000000000001}]},{"osd":2,"last update":"Tue Sep 10 19:01:31 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.51200000000000001,"15min":0.51300000000000001},"min":{"1min":0.39200000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.69099999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.52700000000000002,"15min":0.53300000000000003},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.60599999999999998,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.36099999999999999}]},{"osd":3,"last update":"Tue Sep 10 19:01:31 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.50900000000000001,"15min":0.51100000000000001},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.72799999999999998,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.50900000000000001,"15min":0.51100000000000001},"min":{"1min":0.41199999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.60099999999999998,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.46500000000000002}]},{"osd":5,"last update":"Tue Sep 10 19:01:57 2024","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.35699999999999998,"15min":0.34899999999999998},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.56999999999999995,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.35899999999999999,"15min":0.34799999999999998},"min":{"1min":0.25700000000000001,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.442}]},{"osd":6,"last update":"Tue Sep 10 19:01:17 2024","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.371,"15min":0.371},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.57599999999999996}]},{"osd":7,"last update":"Tue Sep 10 19:01:39 2024","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.56599999999999995}]}]},{"osd":3,"up_from":26,"seq":111669149733,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297988,"kb_used_data":632,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447164,"statfs":{"total":95995035648,"available":95689895936,"internally_reserved":0,"allocated":647168,"data_stored":235664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 19:01:12 2024","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.40200000000000002,"15min":0.40699999999999997},"min":{"1min":0.20100000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.28100000000000003},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.40300000000000002,"15min":0.40899999999999997},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.54700000000000004,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.32700000000000001}]},{"osd":1,"last update":"Tue Sep 10 19:01:12 2024","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.42899999999999999,"15min":0.42999999999999999},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.374},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.42699999999999999,"15min":0.42899999999999999},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.34599999999999997}]},{"osd":2,"last update":"Tue Sep 10 19:01:12 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.498,"15min":0.503},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.48699999999999999,"15min":0.5},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.64800000000000002,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.318}]},{"osd":4,"last update":"Tue Sep 10 19:01:33 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.57599999999999996,"15min":0.57999999999999996},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.68500000000000005,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.56000000000000005,"15min":0.56499999999999995},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.52100000000000002}]},{"osd":5,"last update":"Tue Sep 10 19:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.57799999999999996,"15min":0.58399999999999996},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.71699999999999997,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.56200000000000006,"15min":0.56399999999999995},"min":{"1min":0.441,"5min":0.433,"15min":0.433},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.49299999999999999}]},{"osd":6,"last update":"Tue Sep 10 19:01:15 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.63}]},{"osd":7,"last update":"Tue Sep 10 19:01:41 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.502}]}]},{"osd":2,"up_from":18,"seq":77309411371,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297948,"kb_used_data":592,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447204,"statfs":{"total":95995035648,"available":95689936896,"internally_reserved":0,"allocated":606208,"data_stored":235290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 19:01:52 2024","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.371,"15min":0.37},"min":{"1min":0.19700000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.57499999999999996,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.375,"15min":0.379},"min":{"1min":0.253,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.50600000000000001,"5min":0.53800000000000003,"15min":0.53800000000000003},"last":0.40899999999999997}]},{"osd":1,"last update":"Tue Sep 10 19:01:52 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.38500000000000001,"15min":0.379},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.38600000000000001,"15min":0.38},"min":{"1min":0.25600000000000001,"5min":0.245,"15min":0.245},"max":{"1min":0.48799999999999999,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.433}]},{"osd":3,"last update":"Tue Sep 10 19:01:14 2024","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.44900000000000001,"15min":0.45100000000000001},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"last":0.502},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.497,"15min":0.504},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.47699999999999998}]},{"osd":4,"last update":"Tue Sep 10 19:01:33 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.58199999999999996,"15min":0.57999999999999996},"min":{"1min":0.44900000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.55900000000000005}]},{"osd":5,"last update":"Tue Sep 10 19:01:56 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.60499999999999998,"15min":0.60899999999999999},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.875,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.51400000000000001}]},{"osd":6,"last update":"Tue Sep 10 19:01:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.67800000000000005}]},{"osd":7,"last update":"Tue Sep 10 19:01:39 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.52500000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574895,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297996,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447156,"statfs":{"total":95995035648,"available":95689887744,"internally_reserved":0,"allocated":655360,"data_stored":235587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 19:01:36 2024","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.33800000000000002,"15min":0.32200000000000001},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.375,"5min":0.35299999999999998,"15min":0.33300000000000002},"min":{"1min":0.251,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.436}]},{"osd":2,"last update":"Tue Sep 10 19:01:50 2024","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.379,"15min":0.377},"min":{"1min":0.28100000000000003,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.47899999999999998,"5min":0.53500000000000003,"15min":0.53500000000000003},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.40000000000000002,"15min":0.38900000000000001},"min":{"1min":0.245,"5min":0.221,"15min":0.221},"max":{"1min":0.53800000000000003,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.377}]},{"osd":3,"last update":"Tue Sep 10 19:01:11 2024","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.434,"15min":0.439},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.57699999999999996,"5min":0.58599999999999997,"15min":0.58599999999999997},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.45400000000000001,"15min":0.46000000000000002},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.55800000000000005,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.45500000000000002}]},{"osd":4,"last update":"Tue Sep 10 19:01:36 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.54700000000000004,"15min":0.54600000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.55300000000000005,"15min":0.55500000000000005},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.52100000000000002}]},{"osd":5,"last update":"Tue Sep 10 19:02:00 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.55600000000000005,"15min":0.55000000000000004},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.58499999999999996,"15min":0.58399999999999996},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.55500000000000005}]},{"osd":6,"last update":"Tue Sep 10 19:01:15 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.58699999999999997}]},{"osd":7,"last update":"Tue Sep 10 19:01:40 2024","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":7.7619999999999996,"5min":7.7619999999999996,"15min":7.7619999999999996},"last":0.623},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.68700000000000006}]}]},{"osd":0,"up_from":8,"seq":34359738421,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298532,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446620,"statfs":{"total":95995035648,"available":95689338880,"internally_reserved":0,"allocated":1208320,"data_stored":824325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 10 19:01:35 2024","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.34999999999999998,"15min":0.33800000000000002},"min":{"1min":0.23300000000000001,"5min":0.187,"15min":0.187},"max":{"1min":0.498,"5min":0.625,"15min":0.625},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.34399999999999997,"15min":0.33800000000000002},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.45500000000000002,"5min":0.53700000000000003,"15min":0.53700000000000003},"last":0.307}]},{"osd":2,"last update":"Tue Sep 10 19:01:54 2024","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.39200000000000002,"15min":0.39900000000000002},"min":{"1min":0.248,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.497,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.41199999999999998,"15min":0.41999999999999998},"min":{"1min":0.309,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.50800000000000001,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.41599999999999998}]},{"osd":3,"last update":"Tue Sep 10 19:01:16 2024","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.41699999999999998,"15min":0.42699999999999999},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.46899999999999997,"15min":0.47699999999999998},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.58899999999999997,"5min":0.622,"15min":0.622},"last":0.46200000000000002}]},{"osd":4,"last update":"Tue Sep 10 19:01:35 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.53400000000000003,"15min":0.53000000000000003},"min":{"1min":0.45200000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.52600000000000002,"15min":0.52400000000000002},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.44}]},{"osd":5,"last update":"Tue Sep 10 19:01:54 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.56000000000000005,"15min":0.55800000000000005},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.75600000000000001,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.55200000000000005,"15min":0.55000000000000004},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.47999999999999998}]},{"osd":6,"last update":"Tue Sep 10 19:01:16 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.51500000000000001}]},{"osd":7,"last update":"Tue Sep 10 19:01:41 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.53000000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T19:02:13.655 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid d8563d7e-6fa5-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T19:02:13.912 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:02:14.289 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:14 smithi047 ceph-mon[29019]: from='client.24611 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T19:02:14.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:14 smithi047 ceph-mon[29019]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:02:14.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:14 smithi047 ceph-mon[39856]: from='client.24611 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T19:02:14.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:14 smithi047 ceph-mon[39856]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:02:14.592 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:14 smithi149 ceph-mon[33230]: from='client.24611 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T19:02:14.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:14 smithi149 ceph-mon[33230]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:02:15.730 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T19:02:15.732 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-09-10T19:02:16.393 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:16 smithi047 ceph-mon[29019]: pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:02:16.393 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:16 smithi047 ceph-mon[39856]: pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:02:16.420 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2024-09-10T19:02:15.018158+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596188,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":928,"num_read_kb":785,"num_write":543,"num_write_kb":1671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":548,"ondisk_log_size":548,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2385452,"kb_used_data":6612,"kb_used_omap":13,"kb_used_meta":2378738,"kb_avail":747575764,"statfs":{"total":767960285184,"available":765517582336,"internally_reserved":0,"allocated":6770688,"data_stored":3650184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2435828416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002990"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.581570+0000","last_change":"2024-09-10T19:00:57.180451+0000","last_active":"2024-09-10T19:01:03.581570+0000","last_peered":"2024-09-10T19:01:03.581570+0000","last_clean":"2024-09-10T19:01:03.581570+0000","last_became_active":"2024-09-10T19:00:57.180062+0000","last_became_peered":"2024-09-10T19:00:57.180062+0000","last_unstale":"2024-09-10T19:01:03.581570+0000","last_undegraded":"2024-09-10T19:01:03.581570+0000","last_fullsized":"2024-09-10T19:01:03.581570+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_clean_scrub_stamp":"2024-09-10T19:00:56.075080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:01:34.222831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.519310+0000","last_change":"2024-09-10T19:00:51.060398+0000","last_active":"2024-09-10T19:01:03.519310+0000","last_peered":"2024-09-10T19:01:03.519310+0000","last_clean":"2024-09-10T19:01:03.519310+0000","last_became_active":"2024-09-10T19:00:51.059990+0000","last_became_peered":"2024-09-10T19:00:51.059990+0000","last_unstale":"2024-09-10T19:01:03.519310+0000","last_undegraded":"2024-09-10T19:01:03.519310+0000","last_fullsized":"2024-09-10T19:01:03.519310+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_clean_scrub_stamp":"2024-09-10T19:00:50.042140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:12:18.449280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"60'15","reported_seq":41,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.581544+0000","last_change":"2024-09-10T19:00:53.073694+0000","last_active":"2024-09-10T19:01:03.581544+0000","last_peered":"2024-09-10T19:01:03.581544+0000","last_clean":"2024-09-10T19:01:03.581544+0000","last_became_active":"2024-09-10T19:00:53.073198+0000","last_became_peered":"2024-09-10T19:00:53.073198+0000","last_unstale":"2024-09-10T19:01:03.581544+0000","last_undegraded":"2024-09-10T19:01:03.581544+0000","last_fullsized":"2024-09-10T19:01:03.581544+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_clean_scrub_stamp":"2024-09-10T19:00:52.048288+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:16:51.164807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.517187+0000","last_change":"2024-09-10T19:00:55.094707+0000","last_active":"2024-09-10T19:01:03.517187+0000","last_peered":"2024-09-10T19:01:03.517187+0000","last_clean":"2024-09-10T19:01:03.517187+0000","last_became_active":"2024-09-10T19:00:55.094430+0000","last_became_peered":"2024-09-10T19:00:55.094430+0000","last_unstale":"2024-09-10T19:01:03.517187+0000","last_undegraded":"2024-09-10T19:01:03.517187+0000","last_fullsized":"2024-09-10T19:01:03.517187+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_clean_scrub_stamp":"2024-09-10T19:00:54.064181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:23:57.287401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.582831+0000","last_change":"2024-09-10T19:00:51.062082+0000","last_active":"2024-09-10T19:01:03.582831+0000","last_peered":"2024-09-10T19:01:03.582831+0000","last_clean":"2024-09-10T19:01:03.582831+0000","last_became_active":"2024-09-10T19:00:51.061543+0000","last_became_peered":"2024-09-10T19:00:51.061543+0000","last_unstale":"2024-09-10T19:01:03.582831+0000","last_undegraded":"2024-09-10T19:01:03.582831+0000","last_fullsized":"2024-09-10T19:01:03.582831+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_clean_scrub_stamp":"2024-09-10T19:00:50.042140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:41:42.232976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"60'9","reported_seq":32,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.517195+0000","last_change":"2024-09-10T19:00:53.063307+0000","last_active":"2024-09-10T19:01:03.517195+0000","last_peered":"2024-09-10T19:01:03.517195+0000","last_clean":"2024-09-10T19:01:03.517195+0000","last_became_active":"2024-09-10T19:00:53.063011+0000","last_became_peered":"2024-09-10T19:00:53.063011+0000","last_unstale":"2024-09-10T19:01:03.517195+0000","last_undegraded":"2024-09-10T19:01:03.517195+0000","last_fullsized":"2024-09-10T19:01:03.517195+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_clean_scrub_stamp":"2024-09-10T19:00:52.048288+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:45:31.862691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.581272+0000","last_change":"2024-09-10T19:00:55.092719+0000","last_active":"2024-09-10T19:01:03.581272+0000","last_peered":"2024-09-10T19:01:03.581272+0000","last_clean":"2024-09-10T19:01:03.581272+0000","last_became_active":"2024-09-10T19:00:55.092395+0000","last_became_peered":"2024-09-10T19:00:55.092395+0000","last_unstale":"2024-09-10T19:01:03.581272+0000","last_undegraded":"2024-09-10T19:01:03.581272+0000","last_fullsized":"2024-09-10T19:01:03.581272+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_clean_scrub_stamp":"2024-09-10T19:00:54.064181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:33:52.953786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.517336+0000","last_change":"2024-09-10T19:00:57.097214+0000","last_active":"2024-09-10T19:01:03.517336+0000","last_peered":"2024-09-10T19:01:03.517336+0000","last_clean":"2024-09-10T19:01:03.517336+0000","last_became_active":"2024-09-10T19:00:57.096877+0000","last_became_peered":"2024-09-10T19:00:57.096877+0000","last_unstale":"2024-09-10T19:01:03.517336+0000","last_undegraded":"2024-09-10T19:01:03.517336+0000","last_fullsized":"2024-09-10T19:01:03.517336+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_clean_scrub_stamp":"2024-09-10T19:00:56.075080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:04:45.819611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.582864+0000","last_change":"2024-09-10T19:00:51.066666+0000","last_active":"2024-09-10T19:01:03.582864+0000","last_peered":"2024-09-10T19:01:03.582864+0000","last_clean":"2024-09-10T19:01:03.582864+0000","last_became_active":"2024-09-10T19:00:51.060459+0000","last_became_peered":"2024-09-10T19:00:51.060459+0000","last_unstale":"2024-09-10T19:01:03.582864+0000","last_undegraded":"2024-09-10T19:01:03.582864+0000","last_fullsized":"2024-09-10T19:01:03.582864+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_clean_scrub_stamp":"2024-09-10T19:00:50.042140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:58.199866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"60'5","reported_seq":26,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.517349+0000","last_change":"2024-09-10T19:00:53.069292+0000","last_active":"2024-09-10T19:01:03.517349+0000","last_peered":"2024-09-10T19:01:03.517349+0000","last_clean":"2024-09-10T19:01:03.517349+0000","last_became_active":"2024-09-10T19:00:53.068926+0000","last_became_peered":"2024-09-10T19:00:53.068926+0000","last_unstale":"2024-09-10T19:01:03.517349+0000","last_undegraded":"2024-09-10T19:01:03.517349+0000","last_fullsized":"2024-09-10T19:01:03.517349+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_clean_scrub_stamp":"2024-09-10T19:00:52.048288+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:29:46.049574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.581266+0000","last_change":"2024-09-10T19:00:55.090773+0000","last_active":"2024-09-10T19:01:03.581266+0000","last_peered":"2024-09-10T19:01:03.581266+0000","last_clean":"2024-09-10T19:01:03.581266+0000","last_became_active":"2024-09-10T19:00:55.088329+0000","last_became_peered":"2024-09-10T19:00:55.088329+0000","last_unstale":"2024-09-10T19:01:03.581266+0000","last_undegraded":"2024-09-10T19:01:03.581266+0000","last_fullsized":"2024-09-10T19:01:03.581266+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_clean_scrub_stamp":"2024-09-10T19:00:54.064181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:34:20.701575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.582391+0000","last_change":"2024-09-10T19:00:57.101639+0000","last_active":"2024-09-10T19:01:03.582391+0000","last_peered":"2024-09-10T19:01:03.582391+0000","last_clean":"2024-09-10T19:01:03.582391+0000","last_became_active":"2024-09-10T19:00:57.101289+0000","last_became_peered":"2024-09-10T19:00:57.101289+0000","last_unstale":"2024-09-10T19:01:03.582391+0000","last_undegraded":"2024-09-10T19:01:03.582391+0000","last_fullsized":"2024-09-10T19:01:03.582391+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_clean_scrub_stamp":"2024-09-10T19:00:56.075080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:18:11.861317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.582872+0000","last_change":"2024-09-10T19:00:51.068359+0000","last_active":"2024-09-10T19:01:03.582872+0000","last_peered":"2024-09-10T19:01:03.582872+0000","last_clean":"2024-09-10T19:01:03.582872+0000","last_became_active":"2024-09-10T19:00:51.067717+0000","last_became_peered":"2024-09-10T19:00:51.067717+0000","last_unstale":"2024-09-10T19:01:03.582872+0000","last_undegraded":"2024-09-10T19:01:03.582872+0000","last_fullsized":"2024-09-10T19:01:03.582872+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_clean_scrub_stamp":"2024-09-10T19:00:50.042140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:51:43.094466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"60'9","reported_seq":32,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.517646+0000","last_change":"2024-09-10T19:00:53.065093+0000","last_active":"2024-09-10T19:01:03.517646+0000","last_peered":"2024-09-10T19:01:03.517646+0000","last_clean":"2024-09-10T19:01:03.517646+0000","last_became_active":"2024-09-10T19:00:53.064756+0000","last_became_peered":"2024-09-10T19:00:53.064756+0000","last_unstale":"2024-09-10T19:01:03.517646+0000","last_undegraded":"2024-09-10T19:01:03.517646+0000","last_fullsized":"2024-09-10T19:01:03.517646+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_clean_scrub_stamp":"2024-09-10T19:00:52.048288+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:34:56.308621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.582376+0000","last_change":"2024-09-10T19:00:55.089110+0000","last_active":"2024-09-10T19:01:03.582376+0000","last_peered":"2024-09-10T19:01:03.582376+0000","last_clean":"2024-09-10T19:01:03.582376+0000","last_became_active":"2024-09-10T19:00:55.088796+0000","last_became_peered":"2024-09-10T19:00:55.088796+0000","last_unstale":"2024-09-10T19:01:03.582376+0000","last_undegraded":"2024-09-10T19:01:03.582376+0000","last_fullsized":"2024-09-10T19:01:03.582376+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_clean_scrub_stamp":"2024-09-10T19:00:54.064181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:56.957040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.516997+0000","last_change":"2024-09-10T19:00:57.098397+0000","last_active":"2024-09-10T19:01:03.516997+0000","last_peered":"2024-09-10T19:01:03.516997+0000","last_clean":"2024-09-10T19:01:03.516997+0000","last_became_active":"2024-09-10T19:00:57.097566+0000","last_became_peered":"2024-09-10T19:00:57.097566+0000","last_unstale":"2024-09-10T19:01:03.516997+0000","last_undegraded":"2024-09-10T19:01:03.516997+0000","last_fullsized":"2024-09-10T19:01:03.516997+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_clean_scrub_stamp":"2024-09-10T19:00:56.075080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:45.230498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.d","version":"60'17","reported_seq":44,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.517903+0000","last_change":"2024-09-10T19:00:53.069427+0000","last_active":"2024-09-10T19:01:03.517903+0000","last_peered":"2024-09-10T19:01:03.517903+0000","last_clean":"2024-09-10T19:01:03.517903+0000","last_became_active":"2024-09-10T19:00:53.068732+0000","last_became_peered":"2024-09-10T19:00:53.068732+0000","last_unstale":"2024-09-10T19:01:03.517903+0000","last_undegraded":"2024-09-10T19:01:03.517903+0000","last_fullsized":"2024-09-10T19:01:03.517903+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_clean_scrub_stamp":"2024-09-10T19:00:52.048288+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:25:30.165923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.582341+0000","last_change":"2024-09-10T19:00:51.060295+0000","last_active":"2024-09-10T19:01:03.582341+0000","last_peered":"2024-09-10T19:01:03.582341+0000","last_clean":"2024-09-10T19:01:03.582341+0000","last_became_active":"2024-09-10T19:00:51.059971+0000","last_became_peered":"2024-09-10T19:00:51.059971+0000","last_unstale":"2024-09-10T19:01:03.582341+0000","last_undegraded":"2024-09-10T19:01:03.582341+0000","last_fullsized":"2024-09-10T19:01:03.582341+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_clean_scrub_stamp":"2024-09-10T19:00:50.042140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:28.889400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.582545+0000","last_change":"2024-09-10T19:00:55.088922+0000","last_active":"2024-09-10T19:01:03.582545+0000","last_peered":"2024-09-10T19:01:03.582545+0000","last_clean":"2024-09-10T19:01:03.582545+0000","last_became_active":"2024-09-10T19:00:55.088601+0000","last_became_peered":"2024-09-10T19:00:55.088601+0000","last_unstale":"2024-09-10T19:01:03.582545+0000","last_undegraded":"2024-09-10T19:01:03.582545+0000","last_fullsized":"2024-09-10T19:01:03.582545+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_clean_scrub_stamp":"2024-09-10T19:00:54.064181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:57.963253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.519192+0000","last_change":"2024-09-10T19:00:57.088212+0000","last_active":"2024-09-10T19:01:03.519192+0000","last_peered":"2024-09-10T19:01:03.519192+0000","last_clean":"2024-09-10T19:01:03.519192+0000","last_became_active":"2024-09-10T19:00:57.087890+0000","last_became_peered":"2024-09-10T19:00:57.087890+0000","last_unstale":"2024-09-10T19:01:03.519192+0000","last_undegraded":"2024-09-10T19:01:03.519192+0000","last_fullsized":"2024-09-10T19:01:03.519192+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_clean_scrub_stamp":"2024-09-10T19:00:56.075080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:37:05.603419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"60'10","reported_seq":31,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.517585+0000","last_change":"2024-09-10T19:00:53.065010+0000","last_active":"2024-09-10T19:01:03.517585+0000","last_peered":"2024-09-10T19:01:03.517585+0000","last_clean":"2024-09-10T19:01:03.517585+0000","last_became_active":"2024-09-10T19:00:53.064689+0000","last_became_peered":"2024-09-10T19:00:53.064689+0000","last_unstale":"2024-09-10T19:01:03.517585+0000","last_undegraded":"2024-09-10T19:01:03.517585+0000","last_fullsized":"2024-09-10T19:01:03.517585+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_clean_scrub_stamp":"2024-09-10T19:00:52.048288+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:53:17.508552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.581949+0000","last_change":"2024-09-10T19:00:51.062914+0000","last_active":"2024-09-10T19:01:03.581949+0000","last_peered":"2024-09-10T19:01:03.581949+0000","last_clean":"2024-09-10T19:01:03.581949+0000","last_became_active":"2024-09-10T19:00:51.060081+0000","last_became_peered":"2024-09-10T19:00:51.060081+0000","last_unstale":"2024-09-10T19:01:03.581949+0000","last_undegraded":"2024-09-10T19:01:03.581949+0000","last_fullsized":"2024-09-10T19:01:03.581949+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_clean_scrub_stamp":"2024-09-10T19:00:50.042140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:52.738297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"60'8","reported_seq":38,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T19:02:04.171557+0000","last_change":"2024-09-10T19:00:55.088579+0000","last_active":"2024-09-10T19:02:04.171557+0000","last_peered":"2024-09-10T19:02:04.171557+0000","last_clean":"2024-09-10T19:02:04.171557+0000","last_became_active":"2024-09-10T19:00:55.088301+0000","last_became_peered":"2024-09-10T19:00:55.088301+0000","last_unstale":"2024-09-10T19:02:04.171557+0000","last_undegraded":"2024-09-10T19:02:04.171557+0000","last_fullsized":"2024-09-10T19:02:04.171557+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_clean_scrub_stamp":"2024-09-10T19:00:54.064181+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:38:26.702904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.517691+0000","last_change":"2024-09-10T19:00:57.095572+0000","last_active":"2024-09-10T19:01:03.517691+0000","last_peered":"2024-09-10T19:01:03.517691+0000","last_clean":"2024-09-10T19:01:03.517691+0000","last_became_active":"2024-09-10T19:00:57.095309+0000","last_became_peered":"2024-09-10T19:00:57.095309+0000","last_unstale":"2024-09-10T19:01:03.517691+0000","last_undegraded":"2024-09-10T19:01:03.517691+0000","last_fullsized":"2024-09-10T19:01:03.517691+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_clean_scrub_stamp":"2024-09-10T19:00:56.075080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:12:20.359347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"60'15","reported_seq":41,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.583136+0000","last_change":"2024-09-10T19:00:53.075783+0000","last_active":"2024-09-10T19:01:03.583136+0000","last_peered":"2024-09-10T19:01:03.583136+0000","last_clean":"2024-09-10T19:01:03.583136+0000","last_became_active":"2024-09-10T19:00:53.075350+0000","last_became_peered":"2024-09-10T19:00:53.075350+0000","last_unstale":"2024-09-10T19:01:03.583136+0000","last_undegraded":"2024-09-10T19:01:03.583136+0000","last_fullsized":"2024-09-10T19:01:03.583136+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_clean_scrub_stamp":"2024-09-10T19:00:52.048288+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:06:42.843477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.581700+0000","last_change":"2024-09-10T19:00:51.062932+0000","last_active":"2024-09-10T19:01:03.581700+0000","last_peered":"2024-09-10T19:01:03.581700+0000","last_clean":"2024-09-10T19:01:03.581700+0000","last_became_active":"2024-09-10T19:00:51.062715+0000","last_became_peered":"2024-09-10T19:00:51.062715+0000","last_unstale":"2024-09-10T19:01:03.581700+0000","last_undegraded":"2024-09-10T19:01:03.581700+0000","last_fullsized":"2024-09-10T19:01:03.581700+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_clean_scrub_stamp":"2024-09-10T19:00:50.042140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:24:19.538737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"60'8","reported_seq":38,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T19:02:04.171894+0000","last_change":"2024-09-10T19:00:55.095285+0000","last_active":"2024-09-10T19:02:04.171894+0000","last_peered":"2024-09-10T19:02:04.171894+0000","last_clean":"2024-09-10T19:02:04.171894+0000","last_became_active":"2024-09-10T19:00:55.093624+0000","last_became_peered":"2024-09-10T19:00:55.093624+0000","last_unstale":"2024-09-10T19:02:04.171894+0000","last_undegraded":"2024-09-10T19:02:04.171894+0000","last_fullsized":"2024-09-10T19:02:04.171894+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_clean_scrub_stamp":"2024-09-10T19:00:54.064181+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:38:06.008272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.582575+0000","last_change":"2024-09-10T19:00:57.100829+0000","last_active":"2024-09-10T19:01:03.582575+0000","last_peered":"2024-09-10T19:01:03.582575+0000","last_clean":"2024-09-10T19:01:03.582575+0000","last_became_active":"2024-09-10T19:00:57.100557+0000","last_became_peered":"2024-09-10T19:00:57.100557+0000","last_unstale":"2024-09-10T19:01:03.582575+0000","last_undegraded":"2024-09-10T19:01:03.582575+0000","last_fullsized":"2024-09-10T19:01:03.582575+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_clean_scrub_stamp":"2024-09-10T19:00:56.075080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:33:00.791836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"60'11","reported_seq":35,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.517791+0000","last_change":"2024-09-10T19:00:53.065614+0000","last_active":"2024-09-10T19:01:03.517791+0000","last_peered":"2024-09-10T19:01:03.517791+0000","last_clean":"2024-09-10T19:01:03.517791+0000","last_became_active":"2024-09-10T19:00:53.065329+0000","last_became_peered":"2024-09-10T19:00:53.065329+0000","last_unstale":"2024-09-10T19:01:03.517791+0000","last_undegraded":"2024-09-10T19:01:03.517791+0000","last_fullsized":"2024-09-10T19:01:03.517791+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_clean_scrub_stamp":"2024-09-10T19:00:52.048288+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:12:25.850396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.581309+0000","last_change":"2024-09-10T19:00:51.066299+0000","last_active":"2024-09-10T19:01:03.581309+0000","last_peered":"2024-09-10T19:01:03.581309+0000","last_clean":"2024-09-10T19:01:03.581309+0000","last_became_active":"2024-09-10T19:00:51.066028+0000","last_became_peered":"2024-09-10T19:00:51.066028+0000","last_unstale":"2024-09-10T19:01:03.581309+0000","last_undegraded":"2024-09-10T19:01:03.581309+0000","last_fullsized":"2024-09-10T19:01:03.581309+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_clean_scrub_stamp":"2024-09-10T19:00:50.042140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:55:51.248136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.582650+0000","last_change":"2024-09-10T19:00:55.087250+0000","last_active":"2024-09-10T19:01:03.582650+0000","last_peered":"2024-09-10T19:01:03.582650+0000","last_clean":"2024-09-10T19:01:03.582650+0000","last_became_active":"2024-09-10T19:00:55.086777+0000","last_became_peered":"2024-09-10T19:00:55.086777+0000","last_unstale":"2024-09-10T19:01:03.582650+0000","last_undegraded":"2024-09-10T19:01:03.582650+0000","last_fullsized":"2024-09-10T19:01:03.582650+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_clean_scrub_stamp":"2024-09-10T19:00:54.064181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:46:08.875218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.582146+0000","last_change":"2024-09-10T19:00:57.178915+0000","last_active":"2024-09-10T19:01:03.582146+0000","last_peered":"2024-09-10T19:01:03.582146+0000","last_clean":"2024-09-10T19:01:03.582146+0000","last_became_active":"2024-09-10T19:00:57.178295+0000","last_became_peered":"2024-09-10T19:00:57.178295+0000","last_unstale":"2024-09-10T19:01:03.582146+0000","last_undegraded":"2024-09-10T19:01:03.582146+0000","last_fullsized":"2024-09-10T19:01:03.582146+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_clean_scrub_stamp":"2024-09-10T19:00:56.075080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:29:19.875609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"60'14","reported_seq":37,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.517748+0000","last_change":"2024-09-10T19:00:53.063992+0000","last_active":"2024-09-10T19:01:03.517748+0000","last_peered":"2024-09-10T19:01:03.517748+0000","last_clean":"2024-09-10T19:01:03.517748+0000","last_became_active":"2024-09-10T19:00:53.063647+0000","last_became_peered":"2024-09-10T19:00:53.063647+0000","last_unstale":"2024-09-10T19:01:03.517748+0000","last_undegraded":"2024-09-10T19:01:03.517748+0000","last_fullsized":"2024-09-10T19:01:03.517748+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_clean_scrub_stamp":"2024-09-10T19:00:52.048288+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:11:45.504877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.582221+0000","last_change":"2024-09-10T19:00:51.064084+0000","last_active":"2024-09-10T19:01:03.582221+0000","last_peered":"2024-09-10T19:01:03.582221+0000","last_clean":"2024-09-10T19:01:03.582221+0000","last_became_active":"2024-09-10T19:00:51.060461+0000","last_became_peered":"2024-09-10T19:00:51.060461+0000","last_unstale":"2024-09-10T19:01:03.582221+0000","last_undegraded":"2024-09-10T19:01:03.582221+0000","last_fullsized":"2024-09-10T19:01:03.582221+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_clean_scrub_stamp":"2024-09-10T19:00:50.042140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:39:16.021756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"60'8","reported_seq":38,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T19:02:04.171556+0000","last_change":"2024-09-10T19:00:55.088897+0000","last_active":"2024-09-10T19:02:04.171556+0000","last_peered":"2024-09-10T19:02:04.171556+0000","last_clean":"2024-09-10T19:02:04.171556+0000","last_became_active":"2024-09-10T19:00:55.088594+0000","last_became_peered":"2024-09-10T19:00:55.088594+0000","last_unstale":"2024-09-10T19:02:04.171556+0000","last_undegraded":"2024-09-10T19:02:04.171556+0000","last_fullsized":"2024-09-10T19:02:04.171556+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_clean_scrub_stamp":"2024-09-10T19:00:54.064181+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:21.399086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.581560+0000","last_change":"2024-09-10T19:00:57.178594+0000","last_active":"2024-09-10T19:01:03.581560+0000","last_peered":"2024-09-10T19:01:03.581560+0000","last_clean":"2024-09-10T19:01:03.581560+0000","last_became_active":"2024-09-10T19:00:57.178291+0000","last_became_peered":"2024-09-10T19:00:57.178291+0000","last_unstale":"2024-09-10T19:01:03.581560+0000","last_undegraded":"2024-09-10T19:01:03.581560+0000","last_fullsized":"2024-09-10T19:01:03.581560+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_clean_scrub_stamp":"2024-09-10T19:00:56.075080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:32:44.542113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"60'18","reported_seq":48,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.581255+0000","last_change":"2024-09-10T19:00:53.525001+0000","last_active":"2024-09-10T19:01:03.581255+0000","last_peered":"2024-09-10T19:01:03.581255+0000","last_clean":"2024-09-10T19:01:03.581255+0000","last_became_active":"2024-09-10T19:00:53.524689+0000","last_became_peered":"2024-09-10T19:00:53.524689+0000","last_unstale":"2024-09-10T19:01:03.581255+0000","last_undegraded":"2024-09-10T19:01:03.581255+0000","last_fullsized":"2024-09-10T19:01:03.581255+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_clean_scrub_stamp":"2024-09-10T19:00:52.048288+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:17:53.807969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.583022+0000","last_change":"2024-09-10T19:00:51.063467+0000","last_active":"2024-09-10T19:01:03.583022+0000","last_peered":"2024-09-10T19:01:03.583022+0000","last_clean":"2024-09-10T19:01:03.583022+0000","last_became_active":"2024-09-10T19:00:51.063210+0000","last_became_peered":"2024-09-10T19:00:51.063210+0000","last_unstale":"2024-09-10T19:01:03.583022+0000","last_undegraded":"2024-09-10T19:01:03.583022+0000","last_fullsized":"2024-09-10T19:01:03.583022+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_clean_scrub_stamp":"2024-09-10T19:00:50.042140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:51.264503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.517263+0000","last_change":"2024-09-10T19:00:55.090004+0000","last_active":"2024-09-10T19:01:03.517263+0000","last_peered":"2024-09-10T19:01:03.517263+0000","last_clean":"2024-09-10T19:01:03.517263+0000","last_became_active":"2024-09-10T19:00:55.089535+0000","last_became_peered":"2024-09-10T19:00:55.089535+0000","last_unstale":"2024-09-10T19:01:03.517263+0000","last_undegraded":"2024-09-10T19:01:03.517263+0000","last_fullsized":"2024-09-10T19:01:03.517263+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_clean_scrub_stamp":"2024-09-10T19:00:54.064181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:29:02.951124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.517723+0000","last_change":"2024-09-10T19:00:57.087842+0000","last_active":"2024-09-10T19:01:03.517723+0000","last_peered":"2024-09-10T19:01:03.517723+0000","last_clean":"2024-09-10T19:01:03.517723+0000","last_became_active":"2024-09-10T19:00:57.087580+0000","last_became_peered":"2024-09-10T19:00:57.087580+0000","last_unstale":"2024-09-10T19:01:03.517723+0000","last_undegraded":"2024-09-10T19:01:03.517723+0000","last_fullsized":"2024-09-10T19:01:03.517723+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_clean_scrub_stamp":"2024-09-10T19:00:56.075080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:17.300293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"60'19","reported_seq":52,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.517739+0000","last_change":"2024-09-10T19:00:53.523951+0000","last_active":"2024-09-10T19:01:03.517739+0000","last_peered":"2024-09-10T19:01:03.517739+0000","last_clean":"2024-09-10T19:01:03.517739+0000","last_became_active":"2024-09-10T19:00:53.523676+0000","last_became_peered":"2024-09-10T19:00:53.523676+0000","last_unstale":"2024-09-10T19:01:03.517739+0000","last_undegraded":"2024-09-10T19:01:03.517739+0000","last_fullsized":"2024-09-10T19:01:03.517739+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_clean_scrub_stamp":"2024-09-10T19:00:52.048288+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:13:19.024014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.582930+0000","last_change":"2024-09-10T19:00:51.057836+0000","last_active":"2024-09-10T19:01:03.582930+0000","last_peered":"2024-09-10T19:01:03.582930+0000","last_clean":"2024-09-10T19:01:03.582930+0000","last_became_active":"2024-09-10T19:00:51.057446+0000","last_became_peered":"2024-09-10T19:00:51.057446+0000","last_unstale":"2024-09-10T19:01:03.582930+0000","last_undegraded":"2024-09-10T19:01:03.582930+0000","last_fullsized":"2024-09-10T19:01:03.582930+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_clean_scrub_stamp":"2024-09-10T19:00:50.042140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:16:43.741187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.581290+0000","last_change":"2024-09-10T19:00:55.086455+0000","last_active":"2024-09-10T19:01:03.581290+0000","last_peered":"2024-09-10T19:01:03.581290+0000","last_clean":"2024-09-10T19:01:03.581290+0000","last_became_active":"2024-09-10T19:00:55.086145+0000","last_became_peered":"2024-09-10T19:00:55.086145+0000","last_unstale":"2024-09-10T19:01:03.581290+0000","last_undegraded":"2024-09-10T19:01:03.581290+0000","last_fullsized":"2024-09-10T19:01:03.581290+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_clean_scrub_stamp":"2024-09-10T19:00:54.064181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:23:56.102156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.581187+0000","last_change":"2024-09-10T19:00:57.178515+0000","last_active":"2024-09-10T19:01:03.581187+0000","last_peered":"2024-09-10T19:01:03.581187+0000","last_clean":"2024-09-10T19:01:03.581187+0000","last_became_active":"2024-09-10T19:00:57.178200+0000","last_became_peered":"2024-09-10T19:00:57.178200+0000","last_unstale":"2024-09-10T19:01:03.581187+0000","last_undegraded":"2024-09-10T19:01:03.581187+0000","last_fullsized":"2024-09-10T19:01:03.581187+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_clean_scrub_stamp":"2024-09-10T19:00:56.075080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:40:54.285437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"60'10","reported_seq":31,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.581799+0000","last_change":"2024-09-10T19:00:53.066199+0000","last_active":"2024-09-10T19:01:03.581799+0000","last_peered":"2024-09-10T19:01:03.581799+0000","last_clean":"2024-09-10T19:01:03.581799+0000","last_became_active":"2024-09-10T19:00:53.065660+0000","last_became_peered":"2024-09-10T19:00:53.065660+0000","last_unstale":"2024-09-10T19:01:03.581799+0000","last_undegraded":"2024-09-10T19:01:03.581799+0000","last_fullsized":"2024-09-10T19:01:03.581799+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_clean_scrub_stamp":"2024-09-10T19:00:52.048288+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:31.996329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.582791+0000","last_change":"2024-09-10T19:00:51.063105+0000","last_active":"2024-09-10T19:01:03.582791+0000","last_peered":"2024-09-10T19:01:03.582791+0000","last_clean":"2024-09-10T19:01:03.582791+0000","last_became_active":"2024-09-10T19:00:51.059864+0000","last_became_peered":"2024-09-10T19:00:51.059864+0000","last_unstale":"2024-09-10T19:01:03.582791+0000","last_undegraded":"2024-09-10T19:01:03.582791+0000","last_fullsized":"2024-09-10T19:01:03.582791+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_clean_scrub_stamp":"2024-09-10T19:00:50.042140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:01:14.619190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"60'8","reported_seq":38,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T19:02:04.171568+0000","last_change":"2024-09-10T19:00:55.089934+0000","last_active":"2024-09-10T19:02:04.171568+0000","last_peered":"2024-09-10T19:02:04.171568+0000","last_clean":"2024-09-10T19:02:04.171568+0000","last_became_active":"2024-09-10T19:00:55.089650+0000","last_became_peered":"2024-09-10T19:00:55.089650+0000","last_unstale":"2024-09-10T19:02:04.171568+0000","last_undegraded":"2024-09-10T19:02:04.171568+0000","last_fullsized":"2024-09-10T19:02:04.171568+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_clean_scrub_stamp":"2024-09-10T19:00:54.064181+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:03:36.427766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.516817+0000","last_change":"2024-09-10T19:00:57.098224+0000","last_active":"2024-09-10T19:01:03.516817+0000","last_peered":"2024-09-10T19:01:03.516817+0000","last_clean":"2024-09-10T19:01:03.516817+0000","last_became_active":"2024-09-10T19:00:57.096762+0000","last_became_peered":"2024-09-10T19:00:57.096762+0000","last_unstale":"2024-09-10T19:01:03.516817+0000","last_undegraded":"2024-09-10T19:01:03.516817+0000","last_fullsized":"2024-09-10T19:01:03.516817+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_clean_scrub_stamp":"2024-09-10T19:00:56.075080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:42.237137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"60'28","reported_seq":80,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T19:02:04.171566+0000","last_change":"2024-09-10T19:00:53.068106+0000","last_active":"2024-09-10T19:02:04.171566+0000","last_peered":"2024-09-10T19:02:04.171566+0000","last_clean":"2024-09-10T19:02:04.171566+0000","last_became_active":"2024-09-10T19:00:53.067827+0000","last_became_peered":"2024-09-10T19:00:53.067827+0000","last_unstale":"2024-09-10T19:02:04.171566+0000","last_undegraded":"2024-09-10T19:02:04.171566+0000","last_fullsized":"2024-09-10T19:02:04.171566+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_clean_scrub_stamp":"2024-09-10T19:00:52.048288+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:39:17.494401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.582440+0000","last_change":"2024-09-10T19:00:51.065887+0000","last_active":"2024-09-10T19:01:03.582440+0000","last_peered":"2024-09-10T19:01:03.582440+0000","last_clean":"2024-09-10T19:01:03.582440+0000","last_became_active":"2024-09-10T19:00:51.065539+0000","last_became_peered":"2024-09-10T19:00:51.065539+0000","last_unstale":"2024-09-10T19:01:03.582440+0000","last_undegraded":"2024-09-10T19:01:03.582440+0000","last_fullsized":"2024-09-10T19:01:03.582440+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_clean_scrub_stamp":"2024-09-10T19:00:50.042140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:09:49.598245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.518999+0000","last_change":"2024-09-10T19:00:55.089168+0000","last_active":"2024-09-10T19:01:03.518999+0000","last_peered":"2024-09-10T19:01:03.518999+0000","last_clean":"2024-09-10T19:01:03.518999+0000","last_became_active":"2024-09-10T19:00:55.088776+0000","last_became_peered":"2024-09-10T19:00:55.088776+0000","last_unstale":"2024-09-10T19:01:03.518999+0000","last_undegraded":"2024-09-10T19:01:03.518999+0000","last_fullsized":"2024-09-10T19:01:03.518999+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_clean_scrub_stamp":"2024-09-10T19:00:54.064181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:09:03.566361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"60'1","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.582681+0000","last_change":"2024-09-10T19:00:57.093578+0000","last_active":"2024-09-10T19:01:03.582681+0000","last_peered":"2024-09-10T19:01:03.582681+0000","last_clean":"2024-09-10T19:01:03.582681+0000","last_became_active":"2024-09-10T19:00:57.093224+0000","last_became_peered":"2024-09-10T19:00:57.093224+0000","last_unstale":"2024-09-10T19:01:03.582681+0000","last_undegraded":"2024-09-10T19:01:03.582681+0000","last_fullsized":"2024-09-10T19:01:03.582681+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_clean_scrub_stamp":"2024-09-10T19:00:56.075080+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:33:32.775617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"60'13","reported_seq":43,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.583148+0000","last_change":"2024-09-10T19:00:53.075018+0000","last_active":"2024-09-10T19:01:03.583148+0000","last_peered":"2024-09-10T19:01:03.583148+0000","last_clean":"2024-09-10T19:01:03.583148+0000","last_became_active":"2024-09-10T19:00:53.073402+0000","last_became_peered":"2024-09-10T19:00:53.073402+0000","last_unstale":"2024-09-10T19:01:03.583148+0000","last_undegraded":"2024-09-10T19:01:03.583148+0000","last_fullsized":"2024-09-10T19:01:03.583148+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_clean_scrub_stamp":"2024-09-10T19:00:52.048288+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:12:30.101521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"60'5","reported_seq":101,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T19:02:11.255975+0000","last_change":"2024-09-10T19:00:55.085761+0000","last_active":"2024-09-10T19:02:11.255975+0000","last_peered":"2024-09-10T19:02:11.255975+0000","last_clean":"2024-09-10T19:02:11.255975+0000","last_became_active":"2024-09-10T19:00:51.054621+0000","last_became_peered":"2024-09-10T19:00:51.054621+0000","last_unstale":"2024-09-10T19:02:11.255975+0000","last_undegraded":"2024-09-10T19:02:11.255975+0000","last_fullsized":"2024-09-10T19:02:11.255975+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_clean_scrub_stamp":"2024-09-10T19:00:50.042140+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:07:10.164529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000292179,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":75,"num_read_kb":70,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.518179+0000","last_change":"2024-09-10T19:00:51.058933+0000","last_active":"2024-09-10T19:01:03.518179+0000","last_peered":"2024-09-10T19:01:03.518179+0000","last_clean":"2024-09-10T19:01:03.518179+0000","last_became_active":"2024-09-10T19:00:51.058651+0000","last_became_peered":"2024-09-10T19:00:51.058651+0000","last_unstale":"2024-09-10T19:01:03.518179+0000","last_undegraded":"2024-09-10T19:01:03.518179+0000","last_fullsized":"2024-09-10T19:01:03.518179+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_clean_scrub_stamp":"2024-09-10T19:00:50.042140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:28:10.445664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.517376+0000","last_change":"2024-09-10T19:00:55.084077+0000","last_active":"2024-09-10T19:01:03.517376+0000","last_peered":"2024-09-10T19:01:03.517376+0000","last_clean":"2024-09-10T19:01:03.517376+0000","last_became_active":"2024-09-10T19:00:55.083721+0000","last_became_peered":"2024-09-10T19:00:55.083721+0000","last_unstale":"2024-09-10T19:01:03.517376+0000","last_undegraded":"2024-09-10T19:01:03.517376+0000","last_fullsized":"2024-09-10T19:01:03.517376+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_clean_scrub_stamp":"2024-09-10T19:00:54.064181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:16:38.135924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.582474+0000","last_change":"2024-09-10T19:00:57.179341+0000","last_active":"2024-09-10T19:01:03.582474+0000","last_peered":"2024-09-10T19:01:03.582474+0000","last_clean":"2024-09-10T19:01:03.582474+0000","last_became_active":"2024-09-10T19:00:57.178744+0000","last_became_peered":"2024-09-10T19:00:57.178744+0000","last_unstale":"2024-09-10T19:01:03.582474+0000","last_undegraded":"2024-09-10T19:01:03.582474+0000","last_fullsized":"2024-09-10T19:01:03.582474+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_clean_scrub_stamp":"2024-09-10T19:00:56.075080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:30:59.675745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"60'12","reported_seq":34,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.517657+0000","last_change":"2024-09-10T19:00:53.069777+0000","last_active":"2024-09-10T19:01:03.517657+0000","last_peered":"2024-09-10T19:01:03.517657+0000","last_clean":"2024-09-10T19:01:03.517657+0000","last_became_active":"2024-09-10T19:00:53.069483+0000","last_became_peered":"2024-09-10T19:00:53.069483+0000","last_unstale":"2024-09-10T19:01:03.517657+0000","last_undegraded":"2024-09-10T19:01:03.517657+0000","last_fullsized":"2024-09-10T19:01:03.517657+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_clean_scrub_stamp":"2024-09-10T19:00:52.048288+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:02:29.102995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"55'1","reported_seq":24,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.517523+0000","last_change":"2024-09-10T19:00:55.081265+0000","last_active":"2024-09-10T19:01:03.517523+0000","last_peered":"2024-09-10T19:01:03.517523+0000","last_clean":"2024-09-10T19:01:03.517523+0000","last_became_active":"2024-09-10T19:00:51.060401+0000","last_became_peered":"2024-09-10T19:00:51.060401+0000","last_unstale":"2024-09-10T19:01:03.517523+0000","last_undegraded":"2024-09-10T19:01:03.517523+0000","last_fullsized":"2024-09-10T19:01:03.517523+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_clean_scrub_stamp":"2024-09-10T19:00:50.042140+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:23:02.165568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015209259999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.581376+0000","last_change":"2024-09-10T19:00:51.066604+0000","last_active":"2024-09-10T19:01:03.581376+0000","last_peered":"2024-09-10T19:01:03.581376+0000","last_clean":"2024-09-10T19:01:03.581376+0000","last_became_active":"2024-09-10T19:00:51.066342+0000","last_became_peered":"2024-09-10T19:00:51.066342+0000","last_unstale":"2024-09-10T19:01:03.581376+0000","last_undegraded":"2024-09-10T19:01:03.581376+0000","last_fullsized":"2024-09-10T19:01:03.581376+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_clean_scrub_stamp":"2024-09-10T19:00:50.042140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:25:59.085736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.582731+0000","last_change":"2024-09-10T19:00:55.094027+0000","last_active":"2024-09-10T19:01:03.582731+0000","last_peered":"2024-09-10T19:01:03.582731+0000","last_clean":"2024-09-10T19:01:03.582731+0000","last_became_active":"2024-09-10T19:00:55.093729+0000","last_became_peered":"2024-09-10T19:00:55.093729+0000","last_unstale":"2024-09-10T19:01:03.582731+0000","last_undegraded":"2024-09-10T19:01:03.582731+0000","last_fullsized":"2024-09-10T19:01:03.582731+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_clean_scrub_stamp":"2024-09-10T19:00:54.064181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:48:45.962070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.582474+0000","last_change":"2024-09-10T19:00:57.099929+0000","last_active":"2024-09-10T19:01:03.582474+0000","last_peered":"2024-09-10T19:01:03.582474+0000","last_clean":"2024-09-10T19:01:03.582474+0000","last_became_active":"2024-09-10T19:00:57.099659+0000","last_became_peered":"2024-09-10T19:00:57.099659+0000","last_unstale":"2024-09-10T19:01:03.582474+0000","last_undegraded":"2024-09-10T19:01:03.582474+0000","last_fullsized":"2024-09-10T19:01:03.582474+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_clean_scrub_stamp":"2024-09-10T19:00:56.075080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:08:43.226734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"60'14","reported_seq":38,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.582235+0000","last_change":"2024-09-10T19:00:53.524620+0000","last_active":"2024-09-10T19:01:03.582235+0000","last_peered":"2024-09-10T19:01:03.582235+0000","last_clean":"2024-09-10T19:01:03.582235+0000","last_became_active":"2024-09-10T19:00:53.524335+0000","last_became_peered":"2024-09-10T19:00:53.524335+0000","last_unstale":"2024-09-10T19:01:03.582235+0000","last_undegraded":"2024-09-10T19:01:03.582235+0000","last_fullsized":"2024-09-10T19:01:03.582235+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_clean_scrub_stamp":"2024-09-10T19:00:52.048288+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:05:09.467617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"57'2","reported_seq":25,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.581993+0000","last_change":"2024-09-10T19:00:55.079489+0000","last_active":"2024-09-10T19:01:03.581993+0000","last_peered":"2024-09-10T19:01:03.581993+0000","last_clean":"2024-09-10T19:01:03.581993+0000","last_became_active":"2024-09-10T19:00:51.060879+0000","last_became_peered":"2024-09-10T19:00:51.060879+0000","last_unstale":"2024-09-10T19:01:03.581993+0000","last_undegraded":"2024-09-10T19:01:03.581993+0000","last_fullsized":"2024-09-10T19:01:03.581993+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_clean_scrub_stamp":"2024-09-10T19:00:50.042140+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:53:08.915514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030390389999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.582467+0000","last_change":"2024-09-10T19:00:51.063012+0000","last_active":"2024-09-10T19:01:03.582467+0000","last_peered":"2024-09-10T19:01:03.582467+0000","last_clean":"2024-09-10T19:01:03.582467+0000","last_became_active":"2024-09-10T19:00:51.062733+0000","last_became_peered":"2024-09-10T19:00:51.062733+0000","last_unstale":"2024-09-10T19:01:03.582467+0000","last_undegraded":"2024-09-10T19:01:03.582467+0000","last_fullsized":"2024-09-10T19:01:03.582467+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_clean_scrub_stamp":"2024-09-10T19:00:50.042140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:32:42.120072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"62'79","reported_seq":52,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.058735+0000","last_change":"2024-09-10T19:00:39.512067+0000","last_active":"2024-09-10T19:01:29.058735+0000","last_peered":"2024-09-10T19:01:29.058735+0000","last_clean":"2024-09-10T19:01:29.058735+0000","last_became_active":"2024-09-10T19:00:39.505499+0000","last_became_peered":"2024-09-10T19:00:39.505499+0000","last_unstale":"2024-09-10T19:01:29.058735+0000","last_undegraded":"2024-09-10T19:01:29.058735+0000","last_fullsized":"2024-09-10T19:01:29.058735+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:48.679084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:58:48.679084+0000","last_clean_scrub_stamp":"2024-09-10T18:58:48.679084+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:04:33.625322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.582917+0000","last_change":"2024-09-10T19:00:55.084167+0000","last_active":"2024-09-10T19:01:03.582917+0000","last_peered":"2024-09-10T19:01:03.582917+0000","last_clean":"2024-09-10T19:01:03.582917+0000","last_became_active":"2024-09-10T19:00:55.083860+0000","last_became_peered":"2024-09-10T19:00:55.083860+0000","last_unstale":"2024-09-10T19:01:03.582917+0000","last_undegraded":"2024-09-10T19:01:03.582917+0000","last_fullsized":"2024-09-10T19:01:03.582917+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_clean_scrub_stamp":"2024-09-10T19:00:54.064181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:04:42.181455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.582877+0000","last_change":"2024-09-10T19:00:57.101521+0000","last_active":"2024-09-10T19:01:03.582877+0000","last_peered":"2024-09-10T19:01:03.582877+0000","last_clean":"2024-09-10T19:01:03.582877+0000","last_became_active":"2024-09-10T19:00:57.101236+0000","last_became_peered":"2024-09-10T19:00:57.101236+0000","last_unstale":"2024-09-10T19:01:03.582877+0000","last_undegraded":"2024-09-10T19:01:03.582877+0000","last_fullsized":"2024-09-10T19:01:03.582877+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_clean_scrub_stamp":"2024-09-10T19:00:56.075080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:05:20.475946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"60'19","reported_seq":47,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.582052+0000","last_change":"2024-09-10T19:00:53.524380+0000","last_active":"2024-09-10T19:01:03.582052+0000","last_peered":"2024-09-10T19:01:03.582052+0000","last_clean":"2024-09-10T19:01:03.582052+0000","last_became_active":"2024-09-10T19:00:53.523660+0000","last_became_peered":"2024-09-10T19:00:53.523660+0000","last_unstale":"2024-09-10T19:01:03.582052+0000","last_undegraded":"2024-09-10T19:01:03.582052+0000","last_fullsized":"2024-09-10T19:01:03.582052+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_clean_scrub_stamp":"2024-09-10T19:00:52.048288+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:04:20.329259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.519117+0000","last_change":"2024-09-10T19:00:51.060917+0000","last_active":"2024-09-10T19:01:03.519117+0000","last_peered":"2024-09-10T19:01:03.519117+0000","last_clean":"2024-09-10T19:01:03.519117+0000","last_became_active":"2024-09-10T19:00:51.060413+0000","last_became_peered":"2024-09-10T19:00:51.060413+0000","last_unstale":"2024-09-10T19:01:03.519117+0000","last_undegraded":"2024-09-10T19:01:03.519117+0000","last_fullsized":"2024-09-10T19:01:03.519117+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_clean_scrub_stamp":"2024-09-10T19:00:50.042140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:28:39.206225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.518566+0000","last_change":"2024-09-10T19:00:55.086478+0000","last_active":"2024-09-10T19:01:03.518566+0000","last_peered":"2024-09-10T19:01:03.518566+0000","last_clean":"2024-09-10T19:01:03.518566+0000","last_became_active":"2024-09-10T19:00:55.086185+0000","last_became_peered":"2024-09-10T19:00:55.086185+0000","last_unstale":"2024-09-10T19:01:03.518566+0000","last_undegraded":"2024-09-10T19:01:03.518566+0000","last_fullsized":"2024-09-10T19:01:03.518566+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_clean_scrub_stamp":"2024-09-10T19:00:54.064181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:23.935271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.581448+0000","last_change":"2024-09-10T19:00:57.102559+0000","last_active":"2024-09-10T19:01:03.581448+0000","last_peered":"2024-09-10T19:01:03.581448+0000","last_clean":"2024-09-10T19:01:03.581448+0000","last_became_active":"2024-09-10T19:00:57.102271+0000","last_became_peered":"2024-09-10T19:00:57.102271+0000","last_unstale":"2024-09-10T19:01:03.581448+0000","last_undegraded":"2024-09-10T19:01:03.581448+0000","last_fullsized":"2024-09-10T19:01:03.581448+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_clean_scrub_stamp":"2024-09-10T19:00:56.075080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:05:26.236726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"60'9","reported_seq":32,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.517649+0000","last_change":"2024-09-10T19:00:53.070558+0000","last_active":"2024-09-10T19:01:03.517649+0000","last_peered":"2024-09-10T19:01:03.517649+0000","last_clean":"2024-09-10T19:01:03.517649+0000","last_became_active":"2024-09-10T19:00:53.070183+0000","last_became_peered":"2024-09-10T19:00:53.070183+0000","last_unstale":"2024-09-10T19:01:03.517649+0000","last_undegraded":"2024-09-10T19:01:03.517649+0000","last_fullsized":"2024-09-10T19:01:03.517649+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_clean_scrub_stamp":"2024-09-10T19:00:52.048288+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:19:44.165302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.582714+0000","last_change":"2024-09-10T19:00:51.057966+0000","last_active":"2024-09-10T19:01:03.582714+0000","last_peered":"2024-09-10T19:01:03.582714+0000","last_clean":"2024-09-10T19:01:03.582714+0000","last_became_active":"2024-09-10T19:00:51.057450+0000","last_became_peered":"2024-09-10T19:00:51.057450+0000","last_unstale":"2024-09-10T19:01:03.582714+0000","last_undegraded":"2024-09-10T19:01:03.582714+0000","last_fullsized":"2024-09-10T19:01:03.582714+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_clean_scrub_stamp":"2024-09-10T19:00:50.042140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:06:38.881726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.518408+0000","last_change":"2024-09-10T19:00:55.085077+0000","last_active":"2024-09-10T19:01:03.518408+0000","last_peered":"2024-09-10T19:01:03.518408+0000","last_clean":"2024-09-10T19:01:03.518408+0000","last_became_active":"2024-09-10T19:00:55.084767+0000","last_became_peered":"2024-09-10T19:00:55.084767+0000","last_unstale":"2024-09-10T19:01:03.518408+0000","last_undegraded":"2024-09-10T19:01:03.518408+0000","last_fullsized":"2024-09-10T19:01:03.518408+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_clean_scrub_stamp":"2024-09-10T19:00:54.064181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:49:40.743259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.517020+0000","last_change":"2024-09-10T19:00:57.096433+0000","last_active":"2024-09-10T19:01:03.517020+0000","last_peered":"2024-09-10T19:01:03.517020+0000","last_clean":"2024-09-10T19:01:03.517020+0000","last_became_active":"2024-09-10T19:00:57.096137+0000","last_became_peered":"2024-09-10T19:00:57.096137+0000","last_unstale":"2024-09-10T19:01:03.517020+0000","last_undegraded":"2024-09-10T19:01:03.517020+0000","last_fullsized":"2024-09-10T19:01:03.517020+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_clean_scrub_stamp":"2024-09-10T19:00:56.075080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:06:47.792855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"60'15","reported_seq":41,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.581040+0000","last_change":"2024-09-10T19:00:53.524280+0000","last_active":"2024-09-10T19:01:03.581040+0000","last_peered":"2024-09-10T19:01:03.581040+0000","last_clean":"2024-09-10T19:01:03.581040+0000","last_became_active":"2024-09-10T19:00:53.524000+0000","last_became_peered":"2024-09-10T19:00:53.524000+0000","last_unstale":"2024-09-10T19:01:03.581040+0000","last_undegraded":"2024-09-10T19:01:03.581040+0000","last_fullsized":"2024-09-10T19:01:03.581040+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_clean_scrub_stamp":"2024-09-10T19:00:52.048288+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:21:26.121823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"53'1","reported_seq":27,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.519142+0000","last_change":"2024-09-10T19:00:51.059582+0000","last_active":"2024-09-10T19:01:03.519142+0000","last_peered":"2024-09-10T19:01:03.519142+0000","last_clean":"2024-09-10T19:01:03.519142+0000","last_became_active":"2024-09-10T19:00:51.059305+0000","last_became_peered":"2024-09-10T19:00:51.059305+0000","last_unstale":"2024-09-10T19:01:03.519142+0000","last_undegraded":"2024-09-10T19:01:03.519142+0000","last_fullsized":"2024-09-10T19:01:03.519142+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_clean_scrub_stamp":"2024-09-10T19:00:50.042140+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:01:52.779207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"60'8","reported_seq":41,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T19:02:04.171864+0000","last_change":"2024-09-10T19:00:55.087210+0000","last_active":"2024-09-10T19:02:04.171864+0000","last_peered":"2024-09-10T19:02:04.171864+0000","last_clean":"2024-09-10T19:02:04.171864+0000","last_became_active":"2024-09-10T19:00:55.086941+0000","last_became_peered":"2024-09-10T19:00:55.086941+0000","last_unstale":"2024-09-10T19:02:04.171864+0000","last_undegraded":"2024-09-10T19:02:04.171864+0000","last_fullsized":"2024-09-10T19:02:04.171864+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_clean_scrub_stamp":"2024-09-10T19:00:54.064181+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:43:01.581998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.581429+0000","last_change":"2024-09-10T19:00:57.179191+0000","last_active":"2024-09-10T19:01:03.581429+0000","last_peered":"2024-09-10T19:01:03.581429+0000","last_clean":"2024-09-10T19:01:03.581429+0000","last_became_active":"2024-09-10T19:00:57.178513+0000","last_became_peered":"2024-09-10T19:00:57.178513+0000","last_unstale":"2024-09-10T19:01:03.581429+0000","last_undegraded":"2024-09-10T19:01:03.581429+0000","last_fullsized":"2024-09-10T19:01:03.581429+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_clean_scrub_stamp":"2024-09-10T19:00:56.075080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:27:47.999862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"60'12","reported_seq":39,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.517351+0000","last_change":"2024-09-10T19:00:53.073475+0000","last_active":"2024-09-10T19:01:03.517351+0000","last_peered":"2024-09-10T19:01:03.517351+0000","last_clean":"2024-09-10T19:01:03.517351+0000","last_became_active":"2024-09-10T19:00:53.073182+0000","last_became_peered":"2024-09-10T19:00:53.073182+0000","last_unstale":"2024-09-10T19:01:03.517351+0000","last_undegraded":"2024-09-10T19:01:03.517351+0000","last_fullsized":"2024-09-10T19:01:03.517351+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_clean_scrub_stamp":"2024-09-10T19:00:52.048288+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:22:31.186050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"53'2","reported_seq":35,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.517912+0000","last_change":"2024-09-10T19:00:51.063141+0000","last_active":"2024-09-10T19:01:03.517912+0000","last_peered":"2024-09-10T19:01:03.517912+0000","last_clean":"2024-09-10T19:01:03.517912+0000","last_became_active":"2024-09-10T19:00:51.062863+0000","last_became_peered":"2024-09-10T19:00:51.062863+0000","last_unstale":"2024-09-10T19:01:03.517912+0000","last_undegraded":"2024-09-10T19:01:03.517912+0000","last_fullsized":"2024-09-10T19:01:03.517912+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_clean_scrub_stamp":"2024-09-10T19:00:50.042140+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:39.896731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.518263+0000","last_change":"2024-09-10T19:00:55.088366+0000","last_active":"2024-09-10T19:01:03.518263+0000","last_peered":"2024-09-10T19:01:03.518263+0000","last_clean":"2024-09-10T19:01:03.518263+0000","last_became_active":"2024-09-10T19:00:55.088070+0000","last_became_peered":"2024-09-10T19:00:55.088070+0000","last_unstale":"2024-09-10T19:01:03.518263+0000","last_undegraded":"2024-09-10T19:01:03.518263+0000","last_fullsized":"2024-09-10T19:01:03.518263+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_clean_scrub_stamp":"2024-09-10T19:00:54.064181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:11:23.003295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.581739+0000","last_change":"2024-09-10T19:00:57.098466+0000","last_active":"2024-09-10T19:01:03.581739+0000","last_peered":"2024-09-10T19:01:03.581739+0000","last_clean":"2024-09-10T19:01:03.581739+0000","last_became_active":"2024-09-10T19:00:57.098171+0000","last_became_peered":"2024-09-10T19:00:57.098171+0000","last_unstale":"2024-09-10T19:01:03.581739+0000","last_undegraded":"2024-09-10T19:01:03.581739+0000","last_fullsized":"2024-09-10T19:01:03.581739+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_clean_scrub_stamp":"2024-09-10T19:00:56.075080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:19:57.195328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"60'9","reported_seq":32,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.517576+0000","last_change":"2024-09-10T19:00:53.523852+0000","last_active":"2024-09-10T19:01:03.517576+0000","last_peered":"2024-09-10T19:01:03.517576+0000","last_clean":"2024-09-10T19:01:03.517576+0000","last_became_active":"2024-09-10T19:00:53.523509+0000","last_became_peered":"2024-09-10T19:00:53.523509+0000","last_unstale":"2024-09-10T19:01:03.517576+0000","last_undegraded":"2024-09-10T19:01:03.517576+0000","last_fullsized":"2024-09-10T19:01:03.517576+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_clean_scrub_stamp":"2024-09-10T19:00:52.048288+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:29:11.861725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.518341+0000","last_change":"2024-09-10T19:00:51.061438+0000","last_active":"2024-09-10T19:01:03.518341+0000","last_peered":"2024-09-10T19:01:03.518341+0000","last_clean":"2024-09-10T19:01:03.518341+0000","last_became_active":"2024-09-10T19:00:51.061174+0000","last_became_peered":"2024-09-10T19:00:51.061174+0000","last_unstale":"2024-09-10T19:01:03.518341+0000","last_undegraded":"2024-09-10T19:01:03.518341+0000","last_fullsized":"2024-09-10T19:01:03.518341+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_clean_scrub_stamp":"2024-09-10T19:00:50.042140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:25:41.385708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.582575+0000","last_change":"2024-09-10T19:00:55.090438+0000","last_active":"2024-09-10T19:01:03.582575+0000","last_peered":"2024-09-10T19:01:03.582575+0000","last_clean":"2024-09-10T19:01:03.582575+0000","last_became_active":"2024-09-10T19:00:55.090102+0000","last_became_peered":"2024-09-10T19:00:55.090102+0000","last_unstale":"2024-09-10T19:01:03.582575+0000","last_undegraded":"2024-09-10T19:01:03.582575+0000","last_fullsized":"2024-09-10T19:01:03.582575+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_clean_scrub_stamp":"2024-09-10T19:00:54.064181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:32:09.171597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.518642+0000","last_change":"2024-09-10T19:00:57.096878+0000","last_active":"2024-09-10T19:01:03.518642+0000","last_peered":"2024-09-10T19:01:03.518642+0000","last_clean":"2024-09-10T19:01:03.518642+0000","last_became_active":"2024-09-10T19:00:57.096566+0000","last_became_peered":"2024-09-10T19:00:57.096566+0000","last_unstale":"2024-09-10T19:01:03.518642+0000","last_undegraded":"2024-09-10T19:01:03.518642+0000","last_fullsized":"2024-09-10T19:01:03.518642+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_clean_scrub_stamp":"2024-09-10T19:00:56.075080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:30:52.162107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.581569+0000","last_change":"2024-09-10T19:00:51.056880+0000","last_active":"2024-09-10T19:01:03.581569+0000","last_peered":"2024-09-10T19:01:03.581569+0000","last_clean":"2024-09-10T19:01:03.581569+0000","last_became_active":"2024-09-10T19:00:51.056586+0000","last_became_peered":"2024-09-10T19:00:51.056586+0000","last_unstale":"2024-09-10T19:01:03.581569+0000","last_undegraded":"2024-09-10T19:01:03.581569+0000","last_fullsized":"2024-09-10T19:01:03.581569+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_clean_scrub_stamp":"2024-09-10T19:00:50.042140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:14.070058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"60'6","reported_seq":25,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.582714+0000","last_change":"2024-09-10T19:00:53.524929+0000","last_active":"2024-09-10T19:01:03.582714+0000","last_peered":"2024-09-10T19:01:03.582714+0000","last_clean":"2024-09-10T19:01:03.582714+0000","last_became_active":"2024-09-10T19:00:53.524608+0000","last_became_peered":"2024-09-10T19:00:53.524608+0000","last_unstale":"2024-09-10T19:01:03.582714+0000","last_undegraded":"2024-09-10T19:01:03.582714+0000","last_fullsized":"2024-09-10T19:01:03.582714+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_clean_scrub_stamp":"2024-09-10T19:00:52.048288+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:32.528418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.582494+0000","last_change":"2024-09-10T19:00:55.094110+0000","last_active":"2024-09-10T19:01:03.582494+0000","last_peered":"2024-09-10T19:01:03.582494+0000","last_clean":"2024-09-10T19:01:03.582494+0000","last_became_active":"2024-09-10T19:00:55.093711+0000","last_became_peered":"2024-09-10T19:00:55.093711+0000","last_unstale":"2024-09-10T19:01:03.582494+0000","last_undegraded":"2024-09-10T19:01:03.582494+0000","last_fullsized":"2024-09-10T19:01:03.582494+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_clean_scrub_stamp":"2024-09-10T19:00:54.064181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:06.254108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.582009+0000","last_change":"2024-09-10T19:00:57.179506+0000","last_active":"2024-09-10T19:01:03.582009+0000","last_peered":"2024-09-10T19:01:03.582009+0000","last_clean":"2024-09-10T19:01:03.582009+0000","last_became_active":"2024-09-10T19:00:57.179095+0000","last_became_peered":"2024-09-10T19:00:57.179095+0000","last_unstale":"2024-09-10T19:01:03.582009+0000","last_undegraded":"2024-09-10T19:01:03.582009+0000","last_fullsized":"2024-09-10T19:01:03.582009+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_clean_scrub_stamp":"2024-09-10T19:00:56.075080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:48.626244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"60'10","reported_seq":31,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.582452+0000","last_change":"2024-09-10T19:00:53.525248+0000","last_active":"2024-09-10T19:01:03.582452+0000","last_peered":"2024-09-10T19:01:03.582452+0000","last_clean":"2024-09-10T19:01:03.582452+0000","last_became_active":"2024-09-10T19:00:53.524814+0000","last_became_peered":"2024-09-10T19:00:53.524814+0000","last_unstale":"2024-09-10T19:01:03.582452+0000","last_undegraded":"2024-09-10T19:01:03.582452+0000","last_fullsized":"2024-09-10T19:01:03.582452+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_clean_scrub_stamp":"2024-09-10T19:00:52.048288+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:04.038219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.581606+0000","last_change":"2024-09-10T19:00:51.064379+0000","last_active":"2024-09-10T19:01:03.581606+0000","last_peered":"2024-09-10T19:01:03.581606+0000","last_clean":"2024-09-10T19:01:03.581606+0000","last_became_active":"2024-09-10T19:00:51.064125+0000","last_became_peered":"2024-09-10T19:00:51.064125+0000","last_unstale":"2024-09-10T19:01:03.581606+0000","last_undegraded":"2024-09-10T19:01:03.581606+0000","last_fullsized":"2024-09-10T19:01:03.581606+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_clean_scrub_stamp":"2024-09-10T19:00:50.042140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:02:29.351420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"60'8","reported_seq":38,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T19:02:04.171840+0000","last_change":"2024-09-10T19:00:55.094960+0000","last_active":"2024-09-10T19:02:04.171840+0000","last_peered":"2024-09-10T19:02:04.171840+0000","last_clean":"2024-09-10T19:02:04.171840+0000","last_became_active":"2024-09-10T19:00:55.094474+0000","last_became_peered":"2024-09-10T19:00:55.094474+0000","last_unstale":"2024-09-10T19:02:04.171840+0000","last_undegraded":"2024-09-10T19:02:04.171840+0000","last_fullsized":"2024-09-10T19:02:04.171840+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_clean_scrub_stamp":"2024-09-10T19:00:54.064181+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:09.759680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.517202+0000","last_change":"2024-09-10T19:00:57.095841+0000","last_active":"2024-09-10T19:01:03.517202+0000","last_peered":"2024-09-10T19:01:03.517202+0000","last_clean":"2024-09-10T19:01:03.517202+0000","last_became_active":"2024-09-10T19:00:57.095540+0000","last_became_peered":"2024-09-10T19:00:57.095540+0000","last_unstale":"2024-09-10T19:01:03.517202+0000","last_undegraded":"2024-09-10T19:01:03.517202+0000","last_fullsized":"2024-09-10T19:01:03.517202+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_clean_scrub_stamp":"2024-09-10T19:00:56.075080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:16:13.910364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"60'9","reported_seq":32,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.582924+0000","last_change":"2024-09-10T19:00:53.525632+0000","last_active":"2024-09-10T19:01:03.582924+0000","last_peered":"2024-09-10T19:01:03.582924+0000","last_clean":"2024-09-10T19:01:03.582924+0000","last_became_active":"2024-09-10T19:00:53.525306+0000","last_became_peered":"2024-09-10T19:00:53.525306+0000","last_unstale":"2024-09-10T19:01:03.582924+0000","last_undegraded":"2024-09-10T19:01:03.582924+0000","last_fullsized":"2024-09-10T19:01:03.582924+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_clean_scrub_stamp":"2024-09-10T19:00:52.048288+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:52:17.846341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.517816+0000","last_change":"2024-09-10T19:00:51.058623+0000","last_active":"2024-09-10T19:01:03.517816+0000","last_peered":"2024-09-10T19:01:03.517816+0000","last_clean":"2024-09-10T19:01:03.517816+0000","last_became_active":"2024-09-10T19:00:51.058331+0000","last_became_peered":"2024-09-10T19:00:51.058331+0000","last_unstale":"2024-09-10T19:01:03.517816+0000","last_undegraded":"2024-09-10T19:01:03.517816+0000","last_fullsized":"2024-09-10T19:01:03.517816+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_clean_scrub_stamp":"2024-09-10T19:00:50.042140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:50:02.471740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"60'8","reported_seq":38,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T19:02:04.171590+0000","last_change":"2024-09-10T19:00:55.082961+0000","last_active":"2024-09-10T19:02:04.171590+0000","last_peered":"2024-09-10T19:02:04.171590+0000","last_clean":"2024-09-10T19:02:04.171590+0000","last_became_active":"2024-09-10T19:00:55.082689+0000","last_became_peered":"2024-09-10T19:00:55.082689+0000","last_unstale":"2024-09-10T19:02:04.171590+0000","last_undegraded":"2024-09-10T19:02:04.171590+0000","last_fullsized":"2024-09-10T19:02:04.171590+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_clean_scrub_stamp":"2024-09-10T19:00:54.064181+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:43:19.270561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.517678+0000","last_change":"2024-09-10T19:00:57.095311+0000","last_active":"2024-09-10T19:01:03.517678+0000","last_peered":"2024-09-10T19:01:03.517678+0000","last_clean":"2024-09-10T19:01:03.517678+0000","last_became_active":"2024-09-10T19:00:57.095028+0000","last_became_peered":"2024-09-10T19:00:57.095028+0000","last_unstale":"2024-09-10T19:01:03.517678+0000","last_undegraded":"2024-09-10T19:01:03.517678+0000","last_fullsized":"2024-09-10T19:01:03.517678+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_clean_scrub_stamp":"2024-09-10T19:00:56.075080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:19:56.698879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"60'9","reported_seq":32,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.581681+0000","last_change":"2024-09-10T19:00:53.074967+0000","last_active":"2024-09-10T19:01:03.581681+0000","last_peered":"2024-09-10T19:01:03.581681+0000","last_clean":"2024-09-10T19:01:03.581681+0000","last_became_active":"2024-09-10T19:00:53.073099+0000","last_became_peered":"2024-09-10T19:00:53.073099+0000","last_unstale":"2024-09-10T19:01:03.581681+0000","last_undegraded":"2024-09-10T19:01:03.581681+0000","last_fullsized":"2024-09-10T19:01:03.581681+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_clean_scrub_stamp":"2024-09-10T19:00:52.048288+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:44:13.223102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.581970+0000","last_change":"2024-09-10T19:00:51.065518+0000","last_active":"2024-09-10T19:01:03.581970+0000","last_peered":"2024-09-10T19:01:03.581970+0000","last_clean":"2024-09-10T19:01:03.581970+0000","last_became_active":"2024-09-10T19:00:51.065174+0000","last_became_peered":"2024-09-10T19:00:51.065174+0000","last_unstale":"2024-09-10T19:01:03.581970+0000","last_undegraded":"2024-09-10T19:01:03.581970+0000","last_fullsized":"2024-09-10T19:01:03.581970+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_clean_scrub_stamp":"2024-09-10T19:00:50.042140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:52:06.268154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.581825+0000","last_change":"2024-09-10T19:00:55.091574+0000","last_active":"2024-09-10T19:01:03.581825+0000","last_peered":"2024-09-10T19:01:03.581825+0000","last_clean":"2024-09-10T19:01:03.581825+0000","last_became_active":"2024-09-10T19:00:55.090983+0000","last_became_peered":"2024-09-10T19:00:55.090983+0000","last_unstale":"2024-09-10T19:01:03.581825+0000","last_undegraded":"2024-09-10T19:01:03.581825+0000","last_fullsized":"2024-09-10T19:01:03.581825+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_clean_scrub_stamp":"2024-09-10T19:00:54.064181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:38:25.954120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.516810+0000","last_change":"2024-09-10T19:00:57.098571+0000","last_active":"2024-09-10T19:01:03.516810+0000","last_peered":"2024-09-10T19:01:03.516810+0000","last_clean":"2024-09-10T19:01:03.516810+0000","last_became_active":"2024-09-10T19:00:57.097938+0000","last_became_peered":"2024-09-10T19:00:57.097938+0000","last_unstale":"2024-09-10T19:01:03.516810+0000","last_undegraded":"2024-09-10T19:01:03.516810+0000","last_fullsized":"2024-09-10T19:01:03.516810+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_clean_scrub_stamp":"2024-09-10T19:00:56.075080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:19:14.768601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"60'11","reported_seq":35,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.517368+0000","last_change":"2024-09-10T19:00:53.065637+0000","last_active":"2024-09-10T19:01:03.517368+0000","last_peered":"2024-09-10T19:01:03.517368+0000","last_clean":"2024-09-10T19:01:03.517368+0000","last_became_active":"2024-09-10T19:00:53.065288+0000","last_became_peered":"2024-09-10T19:00:53.065288+0000","last_unstale":"2024-09-10T19:01:03.517368+0000","last_undegraded":"2024-09-10T19:01:03.517368+0000","last_fullsized":"2024-09-10T19:01:03.517368+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_clean_scrub_stamp":"2024-09-10T19:00:52.048288+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:00:39.835523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.582374+0000","last_change":"2024-09-10T19:00:51.063187+0000","last_active":"2024-09-10T19:01:03.582374+0000","last_peered":"2024-09-10T19:01:03.582374+0000","last_clean":"2024-09-10T19:01:03.582374+0000","last_became_active":"2024-09-10T19:00:51.062701+0000","last_became_peered":"2024-09-10T19:00:51.062701+0000","last_unstale":"2024-09-10T19:01:03.582374+0000","last_undegraded":"2024-09-10T19:01:03.582374+0000","last_fullsized":"2024-09-10T19:01:03.582374+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_clean_scrub_stamp":"2024-09-10T19:00:50.042140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:04:11.575350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.581506+0000","last_change":"2024-09-10T19:00:55.093170+0000","last_active":"2024-09-10T19:01:03.581506+0000","last_peered":"2024-09-10T19:01:03.581506+0000","last_clean":"2024-09-10T19:01:03.581506+0000","last_became_active":"2024-09-10T19:00:55.092469+0000","last_became_peered":"2024-09-10T19:00:55.092469+0000","last_unstale":"2024-09-10T19:01:03.581506+0000","last_undegraded":"2024-09-10T19:01:03.581506+0000","last_fullsized":"2024-09-10T19:01:03.581506+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_clean_scrub_stamp":"2024-09-10T19:00:54.064181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:58:11.788948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.581562+0000","last_change":"2024-09-10T19:00:57.096682+0000","last_active":"2024-09-10T19:01:03.581562+0000","last_peered":"2024-09-10T19:01:03.581562+0000","last_clean":"2024-09-10T19:01:03.581562+0000","last_became_active":"2024-09-10T19:00:57.096393+0000","last_became_peered":"2024-09-10T19:00:57.096393+0000","last_unstale":"2024-09-10T19:01:03.581562+0000","last_undegraded":"2024-09-10T19:01:03.581562+0000","last_fullsized":"2024-09-10T19:01:03.581562+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_clean_scrub_stamp":"2024-09-10T19:00:56.075080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:58:52.734339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"60'4","reported_seq":22,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.581169+0000","last_change":"2024-09-10T19:00:53.071584+0000","last_active":"2024-09-10T19:01:03.581169+0000","last_peered":"2024-09-10T19:01:03.581169+0000","last_clean":"2024-09-10T19:01:03.581169+0000","last_became_active":"2024-09-10T19:00:53.070210+0000","last_became_peered":"2024-09-10T19:00:53.070210+0000","last_unstale":"2024-09-10T19:01:03.581169+0000","last_undegraded":"2024-09-10T19:01:03.581169+0000","last_fullsized":"2024-09-10T19:01:03.581169+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_clean_scrub_stamp":"2024-09-10T19:00:52.048288+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:55:11.894284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.582648+0000","last_change":"2024-09-10T19:00:51.064601+0000","last_active":"2024-09-10T19:01:03.582648+0000","last_peered":"2024-09-10T19:01:03.582648+0000","last_clean":"2024-09-10T19:01:03.582648+0000","last_became_active":"2024-09-10T19:00:51.064331+0000","last_became_peered":"2024-09-10T19:00:51.064331+0000","last_unstale":"2024-09-10T19:01:03.582648+0000","last_undegraded":"2024-09-10T19:01:03.582648+0000","last_fullsized":"2024-09-10T19:01:03.582648+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_clean_scrub_stamp":"2024-09-10T19:00:50.042140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:06:48.443784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.581340+0000","last_change":"2024-09-10T19:00:55.089871+0000","last_active":"2024-09-10T19:01:03.581340+0000","last_peered":"2024-09-10T19:01:03.581340+0000","last_clean":"2024-09-10T19:01:03.581340+0000","last_became_active":"2024-09-10T19:00:55.089614+0000","last_became_peered":"2024-09-10T19:00:55.089614+0000","last_unstale":"2024-09-10T19:01:03.581340+0000","last_undegraded":"2024-09-10T19:01:03.581340+0000","last_fullsized":"2024-09-10T19:01:03.581340+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_clean_scrub_stamp":"2024-09-10T19:00:54.064181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:23:07.238782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.581159+0000","last_change":"2024-09-10T19:00:57.096479+0000","last_active":"2024-09-10T19:01:03.581159+0000","last_peered":"2024-09-10T19:01:03.581159+0000","last_clean":"2024-09-10T19:01:03.581159+0000","last_became_active":"2024-09-10T19:00:57.096194+0000","last_became_peered":"2024-09-10T19:00:57.096194+0000","last_unstale":"2024-09-10T19:01:03.581159+0000","last_undegraded":"2024-09-10T19:01:03.581159+0000","last_fullsized":"2024-09-10T19:01:03.581159+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_clean_scrub_stamp":"2024-09-10T19:00:56.075080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:26:20.434998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"60'11","reported_seq":35,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.581286+0000","last_change":"2024-09-10T19:00:53.523667+0000","last_active":"2024-09-10T19:01:03.581286+0000","last_peered":"2024-09-10T19:01:03.581286+0000","last_clean":"2024-09-10T19:01:03.581286+0000","last_became_active":"2024-09-10T19:00:53.523357+0000","last_became_peered":"2024-09-10T19:00:53.523357+0000","last_unstale":"2024-09-10T19:01:03.581286+0000","last_undegraded":"2024-09-10T19:01:03.581286+0000","last_fullsized":"2024-09-10T19:01:03.581286+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_clean_scrub_stamp":"2024-09-10T19:00:52.048288+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:16:44.928708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.582147+0000","last_change":"2024-09-10T19:00:51.061211+0000","last_active":"2024-09-10T19:01:03.582147+0000","last_peered":"2024-09-10T19:01:03.582147+0000","last_clean":"2024-09-10T19:01:03.582147+0000","last_became_active":"2024-09-10T19:00:51.060084+0000","last_became_peered":"2024-09-10T19:00:51.060084+0000","last_unstale":"2024-09-10T19:01:03.582147+0000","last_undegraded":"2024-09-10T19:01:03.582147+0000","last_fullsized":"2024-09-10T19:01:03.582147+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_clean_scrub_stamp":"2024-09-10T19:00:50.042140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:26.265240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.581026+0000","last_change":"2024-09-10T19:00:55.087781+0000","last_active":"2024-09-10T19:01:03.581026+0000","last_peered":"2024-09-10T19:01:03.581026+0000","last_clean":"2024-09-10T19:01:03.581026+0000","last_became_active":"2024-09-10T19:00:55.087396+0000","last_became_peered":"2024-09-10T19:00:55.087396+0000","last_unstale":"2024-09-10T19:01:03.581026+0000","last_undegraded":"2024-09-10T19:01:03.581026+0000","last_fullsized":"2024-09-10T19:01:03.581026+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_clean_scrub_stamp":"2024-09-10T19:00:54.064181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:00:02.924581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.581656+0000","last_change":"2024-09-10T19:00:57.179547+0000","last_active":"2024-09-10T19:01:03.581656+0000","last_peered":"2024-09-10T19:01:03.581656+0000","last_clean":"2024-09-10T19:01:03.581656+0000","last_became_active":"2024-09-10T19:00:57.178628+0000","last_became_peered":"2024-09-10T19:00:57.178628+0000","last_unstale":"2024-09-10T19:01:03.581656+0000","last_undegraded":"2024-09-10T19:01:03.581656+0000","last_fullsized":"2024-09-10T19:01:03.581656+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_clean_scrub_stamp":"2024-09-10T19:00:56.075080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:50:12.142176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"60'1","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.582192+0000","last_change":"2024-09-10T19:00:57.099261+0000","last_active":"2024-09-10T19:01:03.582192+0000","last_peered":"2024-09-10T19:01:03.582192+0000","last_clean":"2024-09-10T19:01:03.582192+0000","last_became_active":"2024-09-10T19:00:57.098986+0000","last_became_peered":"2024-09-10T19:00:57.098986+0000","last_unstale":"2024-09-10T19:01:03.582192+0000","last_undegraded":"2024-09-10T19:01:03.582192+0000","last_fullsized":"2024-09-10T19:01:03.582192+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_clean_scrub_stamp":"2024-09-10T19:00:56.075080+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:19:39.659567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.581062+0000","last_change":"2024-09-10T19:00:51.068124+0000","last_active":"2024-09-10T19:01:03.581062+0000","last_peered":"2024-09-10T19:01:03.581062+0000","last_clean":"2024-09-10T19:01:03.581062+0000","last_became_active":"2024-09-10T19:00:51.067818+0000","last_became_peered":"2024-09-10T19:00:51.067818+0000","last_unstale":"2024-09-10T19:01:03.581062+0000","last_undegraded":"2024-09-10T19:01:03.581062+0000","last_fullsized":"2024-09-10T19:01:03.581062+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_clean_scrub_stamp":"2024-09-10T19:00:50.042140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:38:41.252963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"60'10","reported_seq":31,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.517483+0000","last_change":"2024-09-10T19:00:53.526661+0000","last_active":"2024-09-10T19:01:03.517483+0000","last_peered":"2024-09-10T19:01:03.517483+0000","last_clean":"2024-09-10T19:01:03.517483+0000","last_became_active":"2024-09-10T19:00:53.526412+0000","last_became_peered":"2024-09-10T19:00:53.526412+0000","last_unstale":"2024-09-10T19:01:03.517483+0000","last_undegraded":"2024-09-10T19:01:03.517483+0000","last_fullsized":"2024-09-10T19:01:03.517483+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_clean_scrub_stamp":"2024-09-10T19:00:52.048288+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:00:46.966519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"60'8","reported_seq":41,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T19:02:04.171864+0000","last_change":"2024-09-10T19:00:55.091309+0000","last_active":"2024-09-10T19:02:04.171864+0000","last_peered":"2024-09-10T19:02:04.171864+0000","last_clean":"2024-09-10T19:02:04.171864+0000","last_became_active":"2024-09-10T19:00:55.090505+0000","last_became_peered":"2024-09-10T19:00:55.090505+0000","last_unstale":"2024-09-10T19:02:04.171864+0000","last_undegraded":"2024-09-10T19:02:04.171864+0000","last_fullsized":"2024-09-10T19:02:04.171864+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_clean_scrub_stamp":"2024-09-10T19:00:54.064181+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:56:15.209664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.581761+0000","last_change":"2024-09-10T19:00:57.098698+0000","last_active":"2024-09-10T19:01:03.581761+0000","last_peered":"2024-09-10T19:01:03.581761+0000","last_clean":"2024-09-10T19:01:03.581761+0000","last_became_active":"2024-09-10T19:00:57.098407+0000","last_became_peered":"2024-09-10T19:00:57.098407+0000","last_unstale":"2024-09-10T19:01:03.581761+0000","last_undegraded":"2024-09-10T19:01:03.581761+0000","last_fullsized":"2024-09-10T19:01:03.581761+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_clean_scrub_stamp":"2024-09-10T19:00:56.075080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:12:08.606980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.581450+0000","last_change":"2024-09-10T19:00:51.058555+0000","last_active":"2024-09-10T19:01:03.581450+0000","last_peered":"2024-09-10T19:01:03.581450+0000","last_clean":"2024-09-10T19:01:03.581450+0000","last_became_active":"2024-09-10T19:00:51.058245+0000","last_became_peered":"2024-09-10T19:00:51.058245+0000","last_unstale":"2024-09-10T19:01:03.581450+0000","last_undegraded":"2024-09-10T19:01:03.581450+0000","last_fullsized":"2024-09-10T19:01:03.581450+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_clean_scrub_stamp":"2024-09-10T19:00:50.042140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:57.167503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"60'11","reported_seq":35,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.582070+0000","last_change":"2024-09-10T19:00:53.068178+0000","last_active":"2024-09-10T19:01:03.582070+0000","last_peered":"2024-09-10T19:01:03.582070+0000","last_clean":"2024-09-10T19:01:03.582070+0000","last_became_active":"2024-09-10T19:00:53.067863+0000","last_became_peered":"2024-09-10T19:00:53.067863+0000","last_unstale":"2024-09-10T19:01:03.582070+0000","last_undegraded":"2024-09-10T19:01:03.582070+0000","last_fullsized":"2024-09-10T19:01:03.582070+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_clean_scrub_stamp":"2024-09-10T19:00:52.048288+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:57.926892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.517372+0000","last_change":"2024-09-10T19:00:55.086482+0000","last_active":"2024-09-10T19:01:03.517372+0000","last_peered":"2024-09-10T19:01:03.517372+0000","last_clean":"2024-09-10T19:01:03.517372+0000","last_became_active":"2024-09-10T19:00:55.086198+0000","last_became_peered":"2024-09-10T19:00:55.086198+0000","last_unstale":"2024-09-10T19:01:03.517372+0000","last_undegraded":"2024-09-10T19:01:03.517372+0000","last_fullsized":"2024-09-10T19:01:03.517372+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_clean_scrub_stamp":"2024-09-10T19:00:54.064181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:41:24.304338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.517625+0000","last_change":"2024-09-10T19:00:57.178470+0000","last_active":"2024-09-10T19:01:03.517625+0000","last_peered":"2024-09-10T19:01:03.517625+0000","last_clean":"2024-09-10T19:01:03.517625+0000","last_became_active":"2024-09-10T19:00:57.178092+0000","last_became_peered":"2024-09-10T19:00:57.178092+0000","last_unstale":"2024-09-10T19:01:03.517625+0000","last_undegraded":"2024-09-10T19:01:03.517625+0000","last_fullsized":"2024-09-10T19:01:03.517625+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_clean_scrub_stamp":"2024-09-10T19:00:56.075080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:32:40.554756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"53'1","reported_seq":25,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.582184+0000","last_change":"2024-09-10T19:00:51.063736+0000","last_active":"2024-09-10T19:01:03.582184+0000","last_peered":"2024-09-10T19:01:03.582184+0000","last_clean":"2024-09-10T19:01:03.582184+0000","last_became_active":"2024-09-10T19:00:51.060346+0000","last_became_peered":"2024-09-10T19:00:51.060346+0000","last_unstale":"2024-09-10T19:01:03.582184+0000","last_undegraded":"2024-09-10T19:01:03.582184+0000","last_fullsized":"2024-09-10T19:01:03.582184+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_clean_scrub_stamp":"2024-09-10T19:00:50.042140+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:01:10.085465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"60'15","reported_seq":41,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.518861+0000","last_change":"2024-09-10T19:00:53.066353+0000","last_active":"2024-09-10T19:01:03.518861+0000","last_peered":"2024-09-10T19:01:03.518861+0000","last_clean":"2024-09-10T19:01:03.518861+0000","last_became_active":"2024-09-10T19:00:53.065958+0000","last_became_peered":"2024-09-10T19:00:53.065958+0000","last_unstale":"2024-09-10T19:01:03.518861+0000","last_undegraded":"2024-09-10T19:01:03.518861+0000","last_fullsized":"2024-09-10T19:01:03.518861+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_clean_scrub_stamp":"2024-09-10T19:00:52.048288+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:42:46.537217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.582195+0000","last_change":"2024-09-10T19:00:55.093214+0000","last_active":"2024-09-10T19:01:03.582195+0000","last_peered":"2024-09-10T19:01:03.582195+0000","last_clean":"2024-09-10T19:01:03.582195+0000","last_became_active":"2024-09-10T19:00:55.092890+0000","last_became_peered":"2024-09-10T19:00:55.092890+0000","last_unstale":"2024-09-10T19:01:03.582195+0000","last_undegraded":"2024-09-10T19:01:03.582195+0000","last_fullsized":"2024-09-10T19:01:03.582195+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_clean_scrub_stamp":"2024-09-10T19:00:54.064181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:28:15.101444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.582491+0000","last_change":"2024-09-10T19:00:57.178506+0000","last_active":"2024-09-10T19:01:03.582491+0000","last_peered":"2024-09-10T19:01:03.582491+0000","last_clean":"2024-09-10T19:01:03.582491+0000","last_became_active":"2024-09-10T19:00:57.178181+0000","last_became_peered":"2024-09-10T19:00:57.178181+0000","last_unstale":"2024-09-10T19:01:03.582491+0000","last_undegraded":"2024-09-10T19:01:03.582491+0000","last_fullsized":"2024-09-10T19:01:03.582491+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:56.075080+0000","last_clean_scrub_stamp":"2024-09-10T19:00:56.075080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:49.394838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.582069+0000","last_change":"2024-09-10T19:00:51.061326+0000","last_active":"2024-09-10T19:01:03.582069+0000","last_peered":"2024-09-10T19:01:03.582069+0000","last_clean":"2024-09-10T19:01:03.582069+0000","last_became_active":"2024-09-10T19:00:51.059882+0000","last_became_peered":"2024-09-10T19:00:51.059882+0000","last_unstale":"2024-09-10T19:01:03.582069+0000","last_undegraded":"2024-09-10T19:01:03.582069+0000","last_fullsized":"2024-09-10T19:01:03.582069+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:50.042140+0000","last_clean_scrub_stamp":"2024-09-10T19:00:50.042140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:05:04.740122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"60'12","reported_seq":39,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.582180+0000","last_change":"2024-09-10T19:00:53.073818+0000","last_active":"2024-09-10T19:01:03.582180+0000","last_peered":"2024-09-10T19:01:03.582180+0000","last_clean":"2024-09-10T19:01:03.582180+0000","last_became_active":"2024-09-10T19:00:53.073370+0000","last_became_peered":"2024-09-10T19:00:53.073370+0000","last_unstale":"2024-09-10T19:01:03.582180+0000","last_undegraded":"2024-09-10T19:01:03.582180+0000","last_fullsized":"2024-09-10T19:01:03.582180+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:52.048288+0000","last_clean_scrub_stamp":"2024-09-10T19:00:52.048288+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:48:31.098509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:01:03.517498+0000","last_change":"2024-09-10T19:00:55.092447+0000","last_active":"2024-09-10T19:01:03.517498+0000","last_peered":"2024-09-10T19:01:03.517498+0000","last_clean":"2024-09-10T19:01:03.517498+0000","last_became_active":"2024-09-10T19:00:55.092155+0000","last_became_peered":"2024-09-10T19:00:55.092155+0000","last_unstale":"2024-09-10T19:01:03.517498+0000","last_undegraded":"2024-09-10T19:01:03.517498+0000","last_fullsized":"2024-09-10T19:01:03.517498+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:00:54.064181+0000","last_clean_scrub_stamp":"2024-09-10T19:00:54.064181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:45:52.505237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":391,"ondisk_log_size":391,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":75,"num_read_kb":70,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":49,"seq":210453397525,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298552,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446600,"statfs":{"total":95995035648,"available":95689318400,"internally_reserved":0,"allocated":1224704,"data_stored":826192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 19:01:39 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.621,"5min":0.621,"15min":0.621},"last":0.39800000000000002}]},{"osd":1,"last update":"Tue Sep 10 19:01:39 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.46000000000000002}]},{"osd":2,"last update":"Tue Sep 10 19:01:39 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.38},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.38700000000000001}]},{"osd":3,"last update":"Tue Sep 10 19:01:39 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.42999999999999999}]},{"osd":4,"last update":"Tue Sep 10 19:01:39 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.318},{"interface":"front","average":{"1min":0.439,"5min":0.439,"15min":0.439},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.33600000000000002}]},{"osd":5,"last update":"Tue Sep 10 19:01:39 2024","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.371},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.44900000000000001}]},{"osd":6,"last update":"Tue Sep 10 19:01:39 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.44},{"interface":"front","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.47799999999999998}]}]},{"osd":6,"up_from":43,"seq":184683593753,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298524,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446628,"statfs":{"total":95995035648,"available":95689347072,"internally_reserved":0,"allocated":1200128,"data_stored":824717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 19:01:19 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.9399999999999999,"5min":1.9399999999999999,"15min":1.9399999999999999},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":2.484,"5min":2.484,"15min":2.484},"last":0.63400000000000001}]},{"osd":1,"last update":"Tue Sep 10 19:01:19 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":2.9569999999999999,"5min":2.9569999999999999,"15min":2.9569999999999999},"last":0.501},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.8859999999999999,"5min":1.8859999999999999,"15min":1.8859999999999999},"last":0.53200000000000003}]},{"osd":2,"last update":"Tue Sep 10 19:01:19 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.9119999999999999,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":2.5169999999999999,"5min":2.5169999999999999,"15min":2.5169999999999999},"last":0.51200000000000001}]},{"osd":3,"last update":"Tue Sep 10 19:01:19 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":2.9910000000000001,"5min":2.9910000000000001,"15min":2.9910000000000001},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.8520000000000001,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":0.58199999999999996}]},{"osd":4,"last update":"Tue Sep 10 19:01:19 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":3.484,"5min":3.484,"15min":3.484},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":3.9300000000000002,"5min":3.9300000000000002,"15min":3.9300000000000002},"last":0.40600000000000003}]},{"osd":5,"last update":"Tue Sep 10 19:01:19 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":3.2320000000000002,"5min":3.2320000000000002,"15min":3.2320000000000002},"last":0.438},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":3.331,"5min":3.331,"15min":3.331},"last":0.46899999999999997}]},{"osd":7,"last update":"Tue Sep 10 19:01:41 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":4.1680000000000001,"5min":4.1680000000000001,"15min":4.1680000000000001},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":4.2000000000000002,"5min":4.2000000000000002,"15min":4.2000000000000002},"last":0.59799999999999998}]}]},{"osd":5,"up_from":37,"seq":158913789981,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297940,"kb_used_data":584,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447212,"statfs":{"total":95995035648,"available":95689945088,"internally_reserved":0,"allocated":598016,"data_stored":233608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 19:01:54 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.51200000000000001,"15min":0.50800000000000001},"min":{"1min":0.40999999999999998,"5min":0.312,"15min":0.312},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.47799999999999998,"15min":0.47599999999999998},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.42499999999999999}]},{"osd":1,"last update":"Tue Sep 10 19:01:54 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.48899999999999999,"15min":0.48399999999999999},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.50600000000000001,"15min":0.504},"min":{"1min":0.42399999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.67200000000000004,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.36599999999999999}]},{"osd":2,"last update":"Tue Sep 10 19:01:54 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.501,"15min":0.501},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.36199999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.69799999999999995,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.316}]},{"osd":3,"last update":"Tue Sep 10 19:01:54 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.38900000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.437}]},{"osd":4,"last update":"Tue Sep 10 19:01:54 2024","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.377,"15min":0.376},"min":{"1min":0.308,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.47699999999999998,"5min":0.629,"15min":0.629},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.372,"15min":0.36799999999999999},"min":{"1min":0.32100000000000001,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.499,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.34899999999999998}]},{"osd":6,"last update":"Tue Sep 10 19:01:18 2024","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.38}]},{"osd":7,"last update":"Tue Sep 10 19:01:43 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.47199999999999998}]}]},{"osd":4,"up_from":31,"seq":133143986211,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297972,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447180,"statfs":{"total":95995035648,"available":95689912320,"internally_reserved":0,"allocated":630784,"data_stored":234801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 19:01:31 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.32100000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.49199999999999999,"15min":0.49399999999999999},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.69399999999999995,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.50900000000000001}]},{"osd":1,"last update":"Tue Sep 10 19:01:31 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48999999999999999,"15min":0.49099999999999999},"min":{"1min":0.36399999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.504,"15min":0.5},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.45600000000000002}]},{"osd":2,"last update":"Tue Sep 10 19:01:31 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.51200000000000001,"15min":0.51300000000000001},"min":{"1min":0.39200000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.69099999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.52700000000000002,"15min":0.53300000000000003},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.60599999999999998,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.433}]},{"osd":3,"last update":"Tue Sep 10 19:01:31 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.50900000000000001,"15min":0.51100000000000001},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.72799999999999998,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.50900000000000001,"15min":0.51100000000000001},"min":{"1min":0.41199999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.60099999999999998,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.47099999999999997}]},{"osd":5,"last update":"Tue Sep 10 19:01:57 2024","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.35699999999999998,"15min":0.34899999999999998},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.56999999999999995,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.23999999999999999},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.35899999999999999,"15min":0.34799999999999998},"min":{"1min":0.25700000000000001,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.29199999999999998}]},{"osd":6,"last update":"Tue Sep 10 19:01:17 2024","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.371,"15min":0.371},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.254},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.27800000000000002}]},{"osd":7,"last update":"Tue Sep 10 19:01:39 2024","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.33500000000000002}]}]},{"osd":3,"up_from":26,"seq":111669149734,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297988,"kb_used_data":632,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447164,"statfs":{"total":95995035648,"available":95689895936,"internally_reserved":0,"allocated":647168,"data_stored":235664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 19:01:12 2024","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.40200000000000002,"15min":0.40699999999999997},"min":{"1min":0.20100000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.40300000000000002,"15min":0.40899999999999997},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.54700000000000004,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.308}]},{"osd":1,"last update":"Tue Sep 10 19:01:12 2024","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.42899999999999999,"15min":0.42999999999999999},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.42699999999999999,"15min":0.42899999999999999},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.26600000000000001}]},{"osd":2,"last update":"Tue Sep 10 19:01:12 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.498,"15min":0.503},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.48699999999999999,"15min":0.5},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.64800000000000002,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.36099999999999999}]},{"osd":4,"last update":"Tue Sep 10 19:01:33 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.57599999999999996,"15min":0.57999999999999996},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.68500000000000005,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.56000000000000005,"15min":0.56499999999999995},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.69699999999999995}]},{"osd":5,"last update":"Tue Sep 10 19:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.57799999999999996,"15min":0.58399999999999996},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.71699999999999997,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.56200000000000006,"15min":0.56399999999999995},"min":{"1min":0.441,"5min":0.433,"15min":0.433},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.52400000000000002}]},{"osd":6,"last update":"Tue Sep 10 19:01:15 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.65600000000000003}]},{"osd":7,"last update":"Tue Sep 10 19:01:41 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.66400000000000003}]}]},{"osd":2,"up_from":18,"seq":77309411372,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297948,"kb_used_data":592,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447204,"statfs":{"total":95995035648,"available":95689936896,"internally_reserved":0,"allocated":606208,"data_stored":235290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 19:01:52 2024","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.371,"15min":0.37},"min":{"1min":0.19700000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.57499999999999996,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.375,"15min":0.379},"min":{"1min":0.253,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.50600000000000001,"5min":0.53800000000000003,"15min":0.53800000000000003},"last":0.52200000000000002}]},{"osd":1,"last update":"Tue Sep 10 19:01:52 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.38500000000000001,"15min":0.379},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.497},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.38600000000000001,"15min":0.38},"min":{"1min":0.25600000000000001,"5min":0.245,"15min":0.245},"max":{"1min":0.48799999999999999,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.50600000000000001}]},{"osd":3,"last update":"Tue Sep 10 19:01:14 2024","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.44900000000000001,"15min":0.45100000000000001},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.497,"15min":0.504},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.58899999999999997}]},{"osd":4,"last update":"Tue Sep 10 19:01:33 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.58199999999999996,"15min":0.57999999999999996},"min":{"1min":0.44900000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.56100000000000005}]},{"osd":5,"last update":"Tue Sep 10 19:01:56 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.60499999999999998,"15min":0.60899999999999999},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.875,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.67700000000000005}]},{"osd":6,"last update":"Tue Sep 10 19:01:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.70099999999999996}]},{"osd":7,"last update":"Tue Sep 10 19:01:39 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.60599999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574896,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297996,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447156,"statfs":{"total":95995035648,"available":95689887744,"internally_reserved":0,"allocated":655360,"data_stored":235587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 19:01:36 2024","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.33800000000000002,"15min":0.32200000000000001},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.375,"5min":0.35299999999999998,"15min":0.33300000000000002},"min":{"1min":0.251,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.26500000000000001}]},{"osd":2,"last update":"Tue Sep 10 19:01:50 2024","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.379,"15min":0.377},"min":{"1min":0.28100000000000003,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.47899999999999998,"5min":0.53500000000000003,"15min":0.53500000000000003},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.40000000000000002,"15min":0.38900000000000001},"min":{"1min":0.245,"5min":0.221,"15min":0.221},"max":{"1min":0.53800000000000003,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.60699999999999998}]},{"osd":3,"last update":"Tue Sep 10 19:01:11 2024","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.434,"15min":0.439},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.57699999999999996,"5min":0.58599999999999997,"15min":0.58599999999999997},"last":0.44},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.45400000000000001,"15min":0.46000000000000002},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.55800000000000005,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.36199999999999999}]},{"osd":4,"last update":"Tue Sep 10 19:01:36 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.54700000000000004,"15min":0.54600000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.55300000000000005,"15min":0.55500000000000005},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.45900000000000002}]},{"osd":5,"last update":"Tue Sep 10 19:02:00 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.55600000000000005,"15min":0.55000000000000004},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.58499999999999996,"15min":0.58399999999999996},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.76000000000000001}]},{"osd":6,"last update":"Tue Sep 10 19:01:15 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.502}]},{"osd":7,"last update":"Tue Sep 10 19:01:40 2024","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":7.7619999999999996,"5min":7.7619999999999996,"15min":7.7619999999999996},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.56100000000000005}]}]},{"osd":0,"up_from":8,"seq":34359738422,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298532,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446620,"statfs":{"total":95995035648,"available":95689338880,"internally_reserved":0,"allocated":1208320,"data_stored":824325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 10 19:01:35 2024","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.34999999999999998,"15min":0.33800000000000002},"min":{"1min":0.23300000000000001,"5min":0.187,"15min":0.187},"max":{"1min":0.498,"5min":0.625,"15min":0.625},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.34399999999999997,"15min":0.33800000000000002},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.45500000000000002,"5min":0.53700000000000003,"15min":0.53700000000000003},"last":0.44800000000000001}]},{"osd":2,"last update":"Tue Sep 10 19:01:54 2024","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.39200000000000002,"15min":0.39900000000000002},"min":{"1min":0.248,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.497,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.41199999999999998,"15min":0.41999999999999998},"min":{"1min":0.309,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.50800000000000001,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.48599999999999999}]},{"osd":3,"last update":"Tue Sep 10 19:01:16 2024","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.41699999999999998,"15min":0.42699999999999999},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"last":0.498},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.46899999999999997,"15min":0.47699999999999998},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.58899999999999997,"5min":0.622,"15min":0.622},"last":0.54400000000000004}]},{"osd":4,"last update":"Tue Sep 10 19:01:35 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.53400000000000003,"15min":0.53000000000000003},"min":{"1min":0.45200000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.52600000000000002,"15min":0.52400000000000002},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.53600000000000003}]},{"osd":5,"last update":"Tue Sep 10 19:01:54 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.56000000000000005,"15min":0.55800000000000005},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.75600000000000001,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.55200000000000005,"15min":0.55000000000000004},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.65800000000000003}]},{"osd":6,"last update":"Tue Sep 10 19:01:16 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.57499999999999996}]},{"osd":7,"last update":"Tue Sep 10 19:01:41 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.64100000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T19:02:16.426 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-10T19:02:16.426 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-10T19:02:16.427 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-10T19:02:16.427 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid d8563d7e-6fa5-11ef-bcea-c7b262605968 -- ceph health --format=json 2024-09-10T19:02:16.592 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:16 smithi149 ceph-mon[33230]: pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:02:16.683 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:02:17.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:17 smithi047 ceph-mon[29019]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T19:02:17.539 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:17 smithi047 ceph-mon[39856]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T19:02:17.592 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:17 smithi149 ceph-mon[33230]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T19:02:18.342 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:18 smithi149 ceph-mon[33230]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:02:18.343 INFO:journalctl@ceph.iscsi.iscsi.a.smithi149.stdout:Sep 10 19:02:18 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-iscsi-iscsi-a[60538]: debug there is no tcmu-runner data available 2024-09-10T19:02:18.409 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:18 smithi047 ceph-mon[29019]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:02:18.410 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:18 smithi047 ceph-mon[39856]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:02:18.531 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T19:02:18.531 INFO:teuthology.orchestra.run.smithi047.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-10T19:02:19.350 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-10T19:02:19.350 INFO:tasks.cephadm:Setup complete, yielding 2024-09-10T19:02:19.350 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-10T19:02:19.362 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi047.front.sepia.ceph.com 2024-09-10T19:02:19.363 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d8563d7e-6fa5-11ef-bcea-c7b262605968 -- bash -c 'set -ex 2024-09-10T19:02:19.363 DEBUG:teuthology.orchestra.run.smithi047:> for f in osd.0 osd.1 osd.2 osd.3 osd.4 osd.5 osd.6 osd.7 mgr.y mgr.x 2024-09-10T19:02:19.363 DEBUG:teuthology.orchestra.run.smithi047:> do 2024-09-10T19:02:19.363 DEBUG:teuthology.orchestra.run.smithi047:> echo "rotating key for $f" 2024-09-10T19:02:19.363 DEBUG:teuthology.orchestra.run.smithi047:> K=$(ceph auth get-key $f) 2024-09-10T19:02:19.363 DEBUG:teuthology.orchestra.run.smithi047:> NK="$K" 2024-09-10T19:02:19.364 DEBUG:teuthology.orchestra.run.smithi047:> ceph orch daemon rotate-key $f 2024-09-10T19:02:19.364 DEBUG:teuthology.orchestra.run.smithi047:> while [ "$K" == "$NK" ]; do 2024-09-10T19:02:19.364 DEBUG:teuthology.orchestra.run.smithi047:> sleep 5 2024-09-10T19:02:19.364 DEBUG:teuthology.orchestra.run.smithi047:> NK=$(ceph auth get-key $f) 2024-09-10T19:02:19.364 DEBUG:teuthology.orchestra.run.smithi047:> done 2024-09-10T19:02:19.364 DEBUG:teuthology.orchestra.run.smithi047:> done 2024-09-10T19:02:19.364 DEBUG:teuthology.orchestra.run.smithi047:> ' 2024-09-10T19:02:19.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:19 smithi047 ceph-mon[29019]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:02:19.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:19 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/3627481443' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-10T19:02:19.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:19 smithi047 ceph-mon[39856]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:02:19.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:19 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/3627481443' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-10T19:02:19.592 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:19 smithi149 ceph-mon[33230]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:02:19.592 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:19 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/3627481443' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-10T19:02:19.629 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:02:20.289 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:20 smithi047 ceph-mon[29019]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:02:20.289 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:20 smithi047 ceph-mon[39856]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:02:20.592 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:20 smithi149 ceph-mon[33230]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:02:21.191 INFO:teuthology.orchestra.run.smithi047.stdout:rotating key for osd.0 2024-09-10T19:02:21.192 INFO:teuthology.orchestra.run.smithi047.stderr:+ for f in osd.0 osd.1 osd.2 osd.3 osd.4 osd.5 osd.6 osd.7 mgr.y mgr.x 2024-09-10T19:02:21.192 INFO:teuthology.orchestra.run.smithi047.stderr:+ echo 'rotating key for osd.0' 2024-09-10T19:02:21.192 INFO:teuthology.orchestra.run.smithi047.stderr:++ ceph auth get-key osd.0 2024-09-10T19:02:21.289 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Sep 10 19:02:21 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-alertmanager-a[72004]: ts=2024-09-10T19:02:21.045Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.00201126s 2024-09-10T19:02:21.436 INFO:teuthology.orchestra.run.smithi047.stderr:+ K=AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== 2024-09-10T19:02:21.436 INFO:teuthology.orchestra.run.smithi047.stderr:+ NK=AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== 2024-09-10T19:02:21.436 INFO:teuthology.orchestra.run.smithi047.stderr:+ ceph orch daemon rotate-key osd.0 2024-09-10T19:02:21.709 INFO:teuthology.orchestra.run.smithi047.stdout:Scheduled to rotate-key osd.0 on host 'smithi047' 2024-09-10T19:02:21.720 INFO:teuthology.orchestra.run.smithi047.stderr:+ '[' AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== == AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== ']' 2024-09-10T19:02:21.720 INFO:teuthology.orchestra.run.smithi047.stderr:+ sleep 5 2024-09-10T19:02:22.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:22 smithi047 ceph-mon[29019]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:02:22.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:22 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/3491857205' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:02:22.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:22 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:22.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:22 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:22.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:22 smithi047 ceph-mon[39856]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:02:22.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:22 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/3491857205' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:02:22.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:22 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:22.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:22 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:22.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:22 smithi149 ceph-mon[33230]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:02:22.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:22 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/3491857205' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:02:22.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:22 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:22.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:22 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:22.982 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:02:22 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: ::ffff:172.21.15.149 - - [10/Sep/2024:19:02:22] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:02:23.285 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:23 smithi149 ceph-mon[33230]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:02:23.285 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:23 smithi149 ceph-mon[33230]: Schedule rotate-key daemon osd.0 2024-09-10T19:02:23.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:23 smithi047 ceph-mon[29019]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:02:23.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:23 smithi047 ceph-mon[29019]: Schedule rotate-key daemon osd.0 2024-09-10T19:02:23.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:23 smithi047 ceph-mon[39856]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:02:23.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:23 smithi047 ceph-mon[39856]: Schedule rotate-key daemon osd.0 2024-09-10T19:02:24.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:24 smithi149 ceph-mon[33230]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:02:24.343 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:24 smithi149 systemd[1]: Starting Ceph grafana.a for d8563d7e-6fa5-11ef-bcea-c7b262605968... 2024-09-10T19:02:24.539 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:24 smithi047 ceph-mon[39856]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:02:24.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:24 smithi047 ceph-mon[29019]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:02:24.751 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:24 smithi149 podman[63991]: 2024-09-10 19:02:24.387174507 +0000 UTC m=+0.015152882 image pull 954c08fa618887fe9a6fe572ae541197c77e625b05f4193df7898f5ac649ada9 quay.io/ceph/ceph-grafana:9.4.7 2024-09-10T19:02:24.752 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:24 smithi149 podman[63991]: 2024-09-10 19:02:24.523438774 +0000 UTC m=+0.151417143 container create 53b5807bf7ce1a9bfe8ac992be0754db06c7d6875c3372f0cfe6a028746d06e7 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, io.buildah.version=1.32.0, name=ubi8, summary=Grafana Container configured for Ceph mgr/dashboard integration, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, version=8.5, com.redhat.component=ubi8-container, vendor=Red Hat, Inc., maintainer=Paul Cuzner , com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.openshift.tags=base rhel8, vcs-type=git, architecture=x86_64, description=Ceph Grafana Container, io.k8s.display-name=Red Hat Universal Base Image 8, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, io.openshift.expose-services=, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., distribution-scope=public, release=239.1651231664, build-date=2022-04-29T11:48:11.055897) 2024-09-10T19:02:24.752 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:24 smithi149 podman[63991]: 2024-09-10 19:02:24.750974956 +0000 UTC m=+0.378953326 container init 53b5807bf7ce1a9bfe8ac992be0754db06c7d6875c3372f0cfe6a028746d06e7 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a, com.redhat.component=ubi8-container, name=ubi8, summary=Grafana Container configured for Ceph mgr/dashboard integration, release=239.1651231664, io.openshift.expose-services=, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, architecture=x86_64, io.k8s.display-name=Red Hat Universal Base Image 8, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, distribution-scope=public, build-date=2022-04-29T11:48:11.055897, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, io.buildah.version=1.32.0, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, vcs-type=git, version=8.5, maintainer=Paul Cuzner , vendor=Red Hat, Inc., io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., io.openshift.tags=base rhel8, description=Ceph Grafana Container) 2024-09-10T19:02:25.002 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:24 smithi149 podman[63991]: 2024-09-10 19:02:24.754136638 +0000 UTC m=+0.382115009 container start 53b5807bf7ce1a9bfe8ac992be0754db06c7d6875c3372f0cfe6a028746d06e7 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a, io.openshift.tags=base rhel8, architecture=x86_64, description=Ceph Grafana Container, name=ubi8, vendor=Red Hat, Inc., release=239.1651231664, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, maintainer=Paul Cuzner , url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, version=8.5, build-date=2022-04-29T11:48:11.055897, vcs-type=git, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, io.k8s.display-name=Red Hat Universal Base Image 8, com.redhat.component=ubi8-container, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, distribution-scope=public, io.openshift.expose-services=, summary=Grafana Container configured for Ceph mgr/dashboard integration, io.buildah.version=1.32.0) 2024-09-10T19:02:25.002 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:24 smithi149 bash[63991]: 53b5807bf7ce1a9bfe8ac992be0754db06c7d6875c3372f0cfe6a028746d06e7 2024-09-10T19:02:25.002 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:24 smithi149 systemd[1]: Started Ceph grafana.a for d8563d7e-6fa5-11ef-bcea-c7b262605968. 2024-09-10T19:02:25.003 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:24 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=settings t=2024-09-10T19:02:24.912556915Z level=info msg="Starting Grafana" version=9.4.7 commit=4add91f03d branch=HEAD compiled=2023-03-16T23:56:52Z 2024-09-10T19:02:25.003 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:24 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=settings t=2024-09-10T19:02:24.9127531Z level=warn msg="\"sentry\" frontend logging provider is deprecated and will be removed in the next major version. Use \"grafana\" provider instead." 2024-09-10T19:02:25.003 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:24 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=settings t=2024-09-10T19:02:24.912772338Z level=info msg="Config loaded from" file=/usr/share/grafana/conf/defaults.ini 2024-09-10T19:02:25.003 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:24 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=settings t=2024-09-10T19:02:24.91278081Z level=info msg="Config loaded from" file=/etc/grafana/grafana.ini 2024-09-10T19:02:25.003 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:24 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=settings t=2024-09-10T19:02:24.912786905Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_DATA=/var/lib/grafana" 2024-09-10T19:02:25.003 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:24 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=settings t=2024-09-10T19:02:24.912793091Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_LOGS=/var/log/grafana" 2024-09-10T19:02:25.003 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:24 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=settings t=2024-09-10T19:02:24.912798881Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PLUGINS=/var/lib/grafana/plugins" 2024-09-10T19:02:25.003 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:24 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=settings t=2024-09-10T19:02:24.912806247Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PROVISIONING=/etc/grafana/provisioning" 2024-09-10T19:02:25.003 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:24 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=settings t=2024-09-10T19:02:24.912812218Z level=info msg="Path Home" path=/usr/share/grafana 2024-09-10T19:02:25.003 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:24 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=settings t=2024-09-10T19:02:24.912818086Z level=info msg="Path Data" path=/var/lib/grafana 2024-09-10T19:02:25.003 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:24 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=settings t=2024-09-10T19:02:24.912824205Z level=info msg="Path Logs" path=/var/log/grafana 2024-09-10T19:02:25.003 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:24 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=settings t=2024-09-10T19:02:24.912829835Z level=info msg="Path Plugins" path=/var/lib/grafana/plugins 2024-09-10T19:02:25.003 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:24 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=settings t=2024-09-10T19:02:24.912835509Z level=info msg="Path Provisioning" path=/etc/grafana/provisioning 2024-09-10T19:02:25.004 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:24 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=settings t=2024-09-10T19:02:24.912863667Z level=info msg="App mode production" 2024-09-10T19:02:25.004 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:24 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=sqlstore t=2024-09-10T19:02:24.912915378Z level=info msg="Connecting to DB" dbtype=sqlite3 2024-09-10T19:02:25.004 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:24 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=sqlstore t=2024-09-10T19:02:24.912931762Z level=warn msg="SQLite database file has broader permissions than it should" path=/var/lib/grafana/grafana.db mode=-rw-r--r-- expected=-rw-r----- 2024-09-10T19:02:25.004 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:24 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:24.913776157Z level=info msg="Starting DB migrations" 2024-09-10T19:02:25.004 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:24 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:24.913853459Z level=info msg="Executing migration" id="create migration_log table" 2024-09-10T19:02:25.004 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:24 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:24.914361258Z level=info msg="Executing migration" id="create user table" 2024-09-10T19:02:25.004 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:24 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:24.914799089Z level=info msg="Executing migration" id="add unique index user.login" 2024-09-10T19:02:25.004 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:24 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:24.915262082Z level=info msg="Executing migration" id="add unique index user.email" 2024-09-10T19:02:25.004 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:24 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:24.915678055Z level=info msg="Executing migration" id="drop index UQE_user_login - v1" 2024-09-10T19:02:25.004 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:24 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:24.916239843Z level=info msg="Executing migration" id="drop index UQE_user_email - v1" 2024-09-10T19:02:25.004 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:24 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:24.916620214Z level=info msg="Executing migration" id="Rename table user to user_v1 - v1" 2024-09-10T19:02:25.004 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:24 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:24.917345343Z level=info msg="Executing migration" id="create user table v2" 2024-09-10T19:02:25.004 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:24 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:24.917732407Z level=info msg="Executing migration" id="create index UQE_user_login - v2" 2024-09-10T19:02:25.004 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:24 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:24.918123844Z level=info msg="Executing migration" id="create index UQE_user_email - v2" 2024-09-10T19:02:25.005 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:24 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:24.918529376Z level=info msg="Executing migration" id="copy data_source v1 to v2" 2024-09-10T19:02:25.005 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:24 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:24.91883692Z level=info msg="Executing migration" id="Drop old table user_v1" 2024-09-10T19:02:25.005 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:24 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:24.919250153Z level=info msg="Executing migration" id="Add column help_flags1 to user table" 2024-09-10T19:02:25.005 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:24 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:24.919666268Z level=info msg="Executing migration" id="Update user table charset" 2024-09-10T19:02:25.005 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:24 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:24.920012467Z level=info msg="Executing migration" id="Add last_seen_at column to user" 2024-09-10T19:02:25.006 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:24 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:24.920479562Z level=info msg="Executing migration" id="Add missing user data" 2024-09-10T19:02:25.006 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:24 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:24.920832451Z level=info msg="Executing migration" id="Add is_disabled column to user" 2024-09-10T19:02:25.006 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:24 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:24.921315541Z level=info msg="Executing migration" id="Add index user.login/user.email" 2024-09-10T19:02:25.006 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:24 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:24.921705814Z level=info msg="Executing migration" id="Add is_service_account column to user" 2024-09-10T19:02:25.006 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:24 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:24.922170408Z level=info msg="Executing migration" id="Update is_service_account column to nullable" 2024-09-10T19:02:25.006 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:24 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:24.923370106Z level=info msg="Executing migration" id="create temp user table v1-7" 2024-09-10T19:02:25.006 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:24 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:24.923836935Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v1-7" 2024-09-10T19:02:25.006 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:24 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:24.924366863Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v1-7" 2024-09-10T19:02:25.006 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:24 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:24.92479221Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v1-7" 2024-09-10T19:02:25.006 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:24 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:24.925219572Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v1-7" 2024-09-10T19:02:25.006 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:24 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:24.925618071Z level=info msg="Executing migration" id="Update temp_user table charset" 2024-09-10T19:02:25.006 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:24 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:24.925882224Z level=info msg="Executing migration" id="drop index IDX_temp_user_email - v1" 2024-09-10T19:02:25.006 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:24 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:24.92627948Z level=info msg="Executing migration" id="drop index IDX_temp_user_org_id - v1" 2024-09-10T19:02:25.007 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:24 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:24.926626266Z level=info msg="Executing migration" id="drop index IDX_temp_user_code - v1" 2024-09-10T19:02:25.007 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:24 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:24.927006683Z level=info msg="Executing migration" id="drop index IDX_temp_user_status - v1" 2024-09-10T19:02:25.007 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:24 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:24.927383215Z level=info msg="Executing migration" id="Rename table temp_user to temp_user_tmp_qwerty - v1" 2024-09-10T19:02:25.007 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:24 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:24.928016359Z level=info msg="Executing migration" id="create temp_user v2" 2024-09-10T19:02:25.007 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:24 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:24.928396288Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v2" 2024-09-10T19:02:25.007 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:24 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:24.929708978Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v2" 2024-09-10T19:02:25.007 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:24 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:24.930231241Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v2" 2024-09-10T19:02:25.007 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:24 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:24.930606449Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v2" 2024-09-10T19:02:25.007 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:24 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:24.930982886Z level=info msg="Executing migration" id="copy temp_user v1 to v2" 2024-09-10T19:02:25.007 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:24 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:24.931305615Z level=info msg="Executing migration" id="drop temp_user_tmp_qwerty" 2024-09-10T19:02:25.007 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:24 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:24.931611486Z level=info msg="Executing migration" id="Set created for temp users that will otherwise prematurely expire" 2024-09-10T19:02:25.007 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:24 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:24.931905115Z level=info msg="Executing migration" id="create star table" 2024-09-10T19:02:25.007 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:24 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:24.932316572Z level=info msg="Executing migration" id="add unique index star.user_id_dashboard_id" 2024-09-10T19:02:25.008 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:24 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:24.932721713Z level=info msg="Executing migration" id="create org table v1" 2024-09-10T19:02:25.008 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:24 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:24.933099276Z level=info msg="Executing migration" id="create index UQE_org_name - v1" 2024-09-10T19:02:25.008 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:24 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:24.933505161Z level=info msg="Executing migration" id="create org_user table v1" 2024-09-10T19:02:25.008 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:24 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:24.933862666Z level=info msg="Executing migration" id="create index IDX_org_user_org_id - v1" 2024-09-10T19:02:25.008 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:24 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:24.93427045Z level=info msg="Executing migration" id="create index UQE_org_user_org_id_user_id - v1" 2024-09-10T19:02:25.008 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:24 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:24.934669448Z level=info msg="Executing migration" id="create index IDX_org_user_user_id - v1" 2024-09-10T19:02:25.008 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:24 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:24.935074969Z level=info msg="Executing migration" id="Update org table charset" 2024-09-10T19:02:25.008 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:24 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:24.938001539Z level=info msg="Executing migration" id="Update org_user table charset" 2024-09-10T19:02:25.008 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:24 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:24.940927069Z level=info msg="Executing migration" id="Migrate all Read Only Viewers to Viewers" 2024-09-10T19:02:25.008 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:24 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:24.943736392Z level=info msg="Executing migration" id="create dashboard table" 2024-09-10T19:02:25.008 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:24 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:24.946580181Z level=info msg="Executing migration" id="add index dashboard.account_id" 2024-09-10T19:02:25.008 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:24 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:24.949357436Z level=info msg="Executing migration" id="add unique index dashboard_account_id_slug" 2024-09-10T19:02:25.008 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:24 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:24.952423946Z level=info msg="Executing migration" id="create dashboard_tag table" 2024-09-10T19:02:25.008 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:24 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:24.955209993Z level=info msg="Executing migration" id="add unique index dashboard_tag.dasboard_id_term" 2024-09-10T19:02:25.009 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:24 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:24.957776903Z level=info msg="Executing migration" id="drop index UQE_dashboard_tag_dashboard_id_term - v1" 2024-09-10T19:02:25.009 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:24 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:24.960571025Z level=info msg="Executing migration" id="Rename table dashboard to dashboard_v1 - v1" 2024-09-10T19:02:25.009 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:24 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:24.963072558Z level=info msg="Executing migration" id="create dashboard v2" 2024-09-10T19:02:25.009 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:24 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:24.965553501Z level=info msg="Executing migration" id="create index IDX_dashboard_org_id - v2" 2024-09-10T19:02:25.009 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:24 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:24.968438766Z level=info msg="Executing migration" id="create index UQE_dashboard_org_id_slug - v2" 2024-09-10T19:02:25.009 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:24 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:24.971025224Z level=info msg="Executing migration" id="copy dashboard v1 to v2" 2024-09-10T19:02:25.009 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:24 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:24.973719344Z level=info msg="Executing migration" id="drop table dashboard_v1" 2024-09-10T19:02:25.009 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:24 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:24.976591517Z level=info msg="Executing migration" id="alter dashboard.data to mediumtext v1" 2024-09-10T19:02:25.009 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:24 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:24.979232414Z level=info msg="Executing migration" id="Add column updated_by in dashboard - v2" 2024-09-10T19:02:25.009 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:24 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:24.981705992Z level=info msg="Executing migration" id="Add column created_by in dashboard - v2" 2024-09-10T19:02:25.009 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:24 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:24.984360406Z level=info msg="Executing migration" id="Add column gnetId in dashboard" 2024-09-10T19:02:25.009 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:24 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:24.986919217Z level=info msg="Executing migration" id="Add index for gnetId in dashboard" 2024-09-10T19:02:25.009 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:24 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:24.989483091Z level=info msg="Executing migration" id="Add column plugin_id in dashboard" 2024-09-10T19:02:25.010 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:24 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:24.992357948Z level=info msg="Executing migration" id="Add index for plugin_id in dashboard" 2024-09-10T19:02:25.010 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:24 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:24.99292487Z level=info msg="Executing migration" id="Add index for dashboard_id in dashboard_tag" 2024-09-10T19:02:25.010 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:24 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:24.995843519Z level=info msg="Executing migration" id="Update dashboard table charset" 2024-09-10T19:02:25.010 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:24 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:24.998640586Z level=info msg="Executing migration" id="Update dashboard_tag table charset" 2024-09-10T19:02:25.010 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.001432761Z level=info msg="Executing migration" id="Add column folder_id in dashboard" 2024-09-10T19:02:25.252 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.004312898Z level=info msg="Executing migration" id="Add column isFolder in dashboard" 2024-09-10T19:02:25.253 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.007099381Z level=info msg="Executing migration" id="Add column has_acl in dashboard" 2024-09-10T19:02:25.253 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.009898511Z level=info msg="Executing migration" id="Add column uid in dashboard" 2024-09-10T19:02:25.253 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.01289463Z level=info msg="Executing migration" id="Update uid column values in dashboard" 2024-09-10T19:02:25.253 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.015770147Z level=info msg="Executing migration" id="Add unique index dashboard_org_id_uid" 2024-09-10T19:02:25.253 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.018759462Z level=info msg="Executing migration" id="Remove unique index org_id_slug" 2024-09-10T19:02:25.253 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.02142166Z level=info msg="Executing migration" id="Update dashboard title length" 2024-09-10T19:02:25.253 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.024227562Z level=info msg="Executing migration" id="Add unique index for dashboard_org_id_title_folder_id" 2024-09-10T19:02:25.253 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.027170212Z level=info msg="Executing migration" id="create dashboard_provisioning" 2024-09-10T19:02:25.253 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.030255716Z level=info msg="Executing migration" id="Rename table dashboard_provisioning to dashboard_provisioning_tmp_qwerty - v1" 2024-09-10T19:02:25.253 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.033170663Z level=info msg="Executing migration" id="create dashboard_provisioning v2" 2024-09-10T19:02:25.253 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.035784332Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id - v2" 2024-09-10T19:02:25.254 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.038527184Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id_name - v2" 2024-09-10T19:02:25.254 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.041266908Z level=info msg="Executing migration" id="copy dashboard_provisioning v1 to v2" 2024-09-10T19:02:25.254 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.04395148Z level=info msg="Executing migration" id="drop dashboard_provisioning_tmp_qwerty" 2024-09-10T19:02:25.254 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.046657715Z level=info msg="Executing migration" id="Add check_sum column" 2024-09-10T19:02:25.254 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.049458635Z level=info msg="Executing migration" id="Add index for dashboard_title" 2024-09-10T19:02:25.254 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.051971211Z level=info msg="Executing migration" id="delete tags for deleted dashboards" 2024-09-10T19:02:25.254 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.054459256Z level=info msg="Executing migration" id="delete stars for deleted dashboards" 2024-09-10T19:02:25.254 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.057110277Z level=info msg="Executing migration" id="Add index for dashboard_is_folder" 2024-09-10T19:02:25.254 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.059758848Z level=info msg="Executing migration" id="Add isPublic for dashboard" 2024-09-10T19:02:25.254 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.062601313Z level=info msg="Executing migration" id="create data_source table" 2024-09-10T19:02:25.254 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.065180409Z level=info msg="Executing migration" id="add index data_source.account_id" 2024-09-10T19:02:25.254 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.067922299Z level=info msg="Executing migration" id="add unique index data_source.account_id_name" 2024-09-10T19:02:25.254 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.070618366Z level=info msg="Executing migration" id="drop index IDX_data_source_account_id - v1" 2024-09-10T19:02:25.254 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.073098012Z level=info msg="Executing migration" id="drop index UQE_data_source_account_id_name - v1" 2024-09-10T19:02:25.255 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.076081273Z level=info msg="Executing migration" id="Rename table data_source to data_source_v1 - v1" 2024-09-10T19:02:25.255 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.078493668Z level=info msg="Executing migration" id="create data_source table v2" 2024-09-10T19:02:25.255 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.079305509Z level=info msg="Executing migration" id="create index IDX_data_source_org_id - v2" 2024-09-10T19:02:25.255 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.080214202Z level=info msg="Executing migration" id="create index UQE_data_source_org_id_name - v2" 2024-09-10T19:02:25.255 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.081131535Z level=info msg="Executing migration" id="Drop old table data_source_v1 #2" 2024-09-10T19:02:25.255 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.082016611Z level=info msg="Executing migration" id="Add column with_credentials" 2024-09-10T19:02:25.255 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.082712973Z level=info msg="Executing migration" id="Add secure json data column" 2024-09-10T19:02:25.255 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.083539389Z level=info msg="Executing migration" id="Update data_source table charset" 2024-09-10T19:02:25.255 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.084371613Z level=info msg="Executing migration" id="Update initial version to 1" 2024-09-10T19:02:25.255 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.085210602Z level=info msg="Executing migration" id="Add read_only data column" 2024-09-10T19:02:25.255 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.086080401Z level=info msg="Executing migration" id="Migrate logging ds to loki ds" 2024-09-10T19:02:25.255 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.086934876Z level=info msg="Executing migration" id="Update json_data with nulls" 2024-09-10T19:02:25.256 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.087756397Z level=info msg="Executing migration" id="Add uid column" 2024-09-10T19:02:25.257 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.088766189Z level=info msg="Executing migration" id="Update uid value" 2024-09-10T19:02:25.257 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.089472363Z level=info msg="Executing migration" id="Add unique index datasource_org_id_uid" 2024-09-10T19:02:25.257 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.090383771Z level=info msg="Executing migration" id="add unique index datasource_org_id_is_default" 2024-09-10T19:02:25.257 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.091071028Z level=info msg="Executing migration" id="create api_key table" 2024-09-10T19:02:25.257 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.09149459Z level=info msg="Executing migration" id="add index api_key.account_id" 2024-09-10T19:02:25.257 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.092384845Z level=info msg="Executing migration" id="add index api_key.key" 2024-09-10T19:02:25.257 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.093226781Z level=info msg="Executing migration" id="add index api_key.account_id_name" 2024-09-10T19:02:25.257 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.094040889Z level=info msg="Executing migration" id="drop index IDX_api_key_account_id - v1" 2024-09-10T19:02:25.257 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.094991163Z level=info msg="Executing migration" id="drop index UQE_api_key_key - v1" 2024-09-10T19:02:25.257 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.095514102Z level=info msg="Executing migration" id="drop index UQE_api_key_account_id_name - v1" 2024-09-10T19:02:25.257 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.096351506Z level=info msg="Executing migration" id="Rename table api_key to api_key_v1 - v1" 2024-09-10T19:02:25.257 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.09750524Z level=info msg="Executing migration" id="create api_key table v2" 2024-09-10T19:02:25.257 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.097975705Z level=info msg="Executing migration" id="create index IDX_api_key_org_id - v2" 2024-09-10T19:02:25.257 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.098800135Z level=info msg="Executing migration" id="create index UQE_api_key_key - v2" 2024-09-10T19:02:25.258 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.099268585Z level=info msg="Executing migration" id="create index UQE_api_key_org_id_name - v2" 2024-09-10T19:02:25.258 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.100166787Z level=info msg="Executing migration" id="copy api_key v1 to v2" 2024-09-10T19:02:25.258 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.100945242Z level=info msg="Executing migration" id="Drop old table api_key_v1" 2024-09-10T19:02:25.258 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.101807991Z level=info msg="Executing migration" id="Update api_key table charset" 2024-09-10T19:02:25.258 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.102138159Z level=info msg="Executing migration" id="Add expires to api_key table" 2024-09-10T19:02:25.258 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.102965264Z level=info msg="Executing migration" id="Add service account foreign key" 2024-09-10T19:02:25.258 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.103571756Z level=info msg="Executing migration" id="set service account foreign key to nil if 0" 2024-09-10T19:02:25.258 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.103882477Z level=info msg="Executing migration" id="Add last_used_at to api_key table" 2024-09-10T19:02:25.258 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.104444918Z level=info msg="Executing migration" id="Add is_revoked column to api_key table" 2024-09-10T19:02:25.258 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.104965597Z level=info msg="Executing migration" id="create dashboard_snapshot table v4" 2024-09-10T19:02:25.258 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.105347828Z level=info msg="Executing migration" id="drop table dashboard_snapshot_v4 #1" 2024-09-10T19:02:25.258 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.108240621Z level=info msg="Executing migration" id="create dashboard_snapshot table v5 #2" 2024-09-10T19:02:25.259 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.108659514Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_key - v5" 2024-09-10T19:02:25.259 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.110968036Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_delete_key - v5" 2024-09-10T19:02:25.259 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.11376108Z level=info msg="Executing migration" id="create index IDX_dashboard_snapshot_user_id - v5" 2024-09-10T19:02:25.259 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.116496548Z level=info msg="Executing migration" id="alter dashboard_snapshot to mediumtext v2" 2024-09-10T19:02:25.259 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.116807921Z level=info msg="Executing migration" id="Update dashboard_snapshot table charset" 2024-09-10T19:02:25.259 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.119736527Z level=info msg="Executing migration" id="Add column external_delete_url to dashboard_snapshots table" 2024-09-10T19:02:25.259 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.122619091Z level=info msg="Executing migration" id="Add encrypted dashboard json column" 2024-09-10T19:02:25.259 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.125266158Z level=info msg="Executing migration" id="Change dashboard_encrypted column to MEDIUMBLOB" 2024-09-10T19:02:25.259 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.125588447Z level=info msg="Executing migration" id="create quota table v1" 2024-09-10T19:02:25.259 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.128674738Z level=info msg="Executing migration" id="create index UQE_quota_org_id_user_id_target - v1" 2024-09-10T19:02:25.260 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.13156014Z level=info msg="Executing migration" id="Update quota table charset" 2024-09-10T19:02:25.260 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.134230232Z level=info msg="Executing migration" id="create plugin_setting table" 2024-09-10T19:02:25.260 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.134679338Z level=info msg="Executing migration" id="create index UQE_plugin_setting_org_id_plugin_id - v1" 2024-09-10T19:02:25.260 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.1377964Z level=info msg="Executing migration" id="Add column plugin_version to plugin_settings" 2024-09-10T19:02:25.260 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.140667108Z level=info msg="Executing migration" id="Update plugin_setting table charset" 2024-09-10T19:02:25.260 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.143329651Z level=info msg="Executing migration" id="create session table" 2024-09-10T19:02:25.260 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.145779697Z level=info msg="Executing migration" id="Drop old table playlist table" 2024-09-10T19:02:25.260 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.148599801Z level=info msg="Executing migration" id="Drop old table playlist_item table" 2024-09-10T19:02:25.260 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.151218903Z level=info msg="Executing migration" id="create playlist table v2" 2024-09-10T19:02:25.260 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.154034139Z level=info msg="Executing migration" id="create playlist item table v2" 2024-09-10T19:02:25.260 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.156671175Z level=info msg="Executing migration" id="Update playlist table charset" 2024-09-10T19:02:25.260 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.159326311Z level=info msg="Executing migration" id="Update playlist_item table charset" 2024-09-10T19:02:25.260 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.162002773Z level=info msg="Executing migration" id="drop preferences table v2" 2024-09-10T19:02:25.260 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.164767781Z level=info msg="Executing migration" id="drop preferences table v3" 2024-09-10T19:02:25.261 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.167518443Z level=info msg="Executing migration" id="create preferences table v3" 2024-09-10T19:02:25.261 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.170219906Z level=info msg="Executing migration" id="Update preferences table charset" 2024-09-10T19:02:25.261 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.172923484Z level=info msg="Executing migration" id="Add column team_id in preferences" 2024-09-10T19:02:25.261 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.175440241Z level=info msg="Executing migration" id="Update team_id column values in preferences" 2024-09-10T19:02:25.261 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.178110358Z level=info msg="Executing migration" id="Add column week_start in preferences" 2024-09-10T19:02:25.261 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.180826029Z level=info msg="Executing migration" id="Add column preferences.json_data" 2024-09-10T19:02:25.261 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.183379141Z level=info msg="Executing migration" id="alter preferences.json_data to mediumtext v1" 2024-09-10T19:02:25.261 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.186262799Z level=info msg="Executing migration" id="Add preferences index org_id" 2024-09-10T19:02:25.261 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.188833395Z level=info msg="Executing migration" id="Add preferences index user_id" 2024-09-10T19:02:25.261 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.189764015Z level=info msg="Executing migration" id="create alert table v1" 2024-09-10T19:02:25.261 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.190522022Z level=info msg="Executing migration" id="add index alert org_id & id " 2024-09-10T19:02:25.261 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.191375413Z level=info msg="Executing migration" id="add index alert state" 2024-09-10T19:02:25.261 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.192224348Z level=info msg="Executing migration" id="add index alert dashboard_id" 2024-09-10T19:02:25.261 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.19318395Z level=info msg="Executing migration" id="Create alert_rule_tag table v1" 2024-09-10T19:02:25.261 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.193902944Z level=info msg="Executing migration" id="Add unique index alert_rule_tag.alert_id_tag_id" 2024-09-10T19:02:25.262 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.194769058Z level=info msg="Executing migration" id="drop index UQE_alert_rule_tag_alert_id_tag_id - v1" 2024-09-10T19:02:25.262 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.195709324Z level=info msg="Executing migration" id="Rename table alert_rule_tag to alert_rule_tag_v1 - v1" 2024-09-10T19:02:25.262 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.197362241Z level=info msg="Executing migration" id="Create alert_rule_tag table v2" 2024-09-10T19:02:25.262 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.198163106Z level=info msg="Executing migration" id="create index UQE_alert_rule_tag_alert_id_tag_id - Add unique index alert_rule_tag.alert_id_tag_id V2" 2024-09-10T19:02:25.262 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.199150292Z level=info msg="Executing migration" id="copy alert_rule_tag v1 to v2" 2024-09-10T19:02:25.262 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.199887318Z level=info msg="Executing migration" id="drop table alert_rule_tag_v1" 2024-09-10T19:02:25.262 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.200742109Z level=info msg="Executing migration" id="create alert_notification table v1" 2024-09-10T19:02:25.262 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.201609599Z level=info msg="Executing migration" id="Add column is_default" 2024-09-10T19:02:25.262 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.202477633Z level=info msg="Executing migration" id="Add column frequency" 2024-09-10T19:02:25.262 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.203337787Z level=info msg="Executing migration" id="Add column send_reminder" 2024-09-10T19:02:25.262 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.204357546Z level=info msg="Executing migration" id="Add column disable_resolve_message" 2024-09-10T19:02:25.262 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.205149399Z level=info msg="Executing migration" id="add index alert_notification org_id & name" 2024-09-10T19:02:25.262 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.205994016Z level=info msg="Executing migration" id="Update alert table charset" 2024-09-10T19:02:25.262 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.206993372Z level=info msg="Executing migration" id="Update alert_notification table charset" 2024-09-10T19:02:25.262 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.207738414Z level=info msg="Executing migration" id="create notification_journal table v1" 2024-09-10T19:02:25.263 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.208626374Z level=info msg="Executing migration" id="add index notification_journal org_id & alert_id & notifier_id" 2024-09-10T19:02:25.263 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.20946938Z level=info msg="Executing migration" id="drop alert_notification_journal" 2024-09-10T19:02:25.263 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.210334683Z level=info msg="Executing migration" id="create alert_notification_state table v1" 2024-09-10T19:02:25.263 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.211175929Z level=info msg="Executing migration" id="add index alert_notification_state org_id & alert_id & notifier_id" 2024-09-10T19:02:25.263 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.212123196Z level=info msg="Executing migration" id="Add for to alert table" 2024-09-10T19:02:25.263 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.212897695Z level=info msg="Executing migration" id="Add column uid in alert_notification" 2024-09-10T19:02:25.263 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.213747276Z level=info msg="Executing migration" id="Update uid column values in alert_notification" 2024-09-10T19:02:25.263 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.2147552Z level=info msg="Executing migration" id="Add unique index alert_notification_org_id_uid" 2024-09-10T19:02:25.263 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.215506169Z level=info msg="Executing migration" id="Remove unique index org_id_name" 2024-09-10T19:02:25.263 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.216414715Z level=info msg="Executing migration" id="Add column secure_settings in alert_notification" 2024-09-10T19:02:25.263 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.217320669Z level=info msg="Executing migration" id="alter alert.settings to mediumtext" 2024-09-10T19:02:25.263 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.218815912Z level=info msg="Executing migration" id="Add non-unique index alert_notification_state_alert_id" 2024-09-10T19:02:25.263 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.221463338Z level=info msg="Executing migration" id="Add non-unique index alert_rule_tag_alert_id" 2024-09-10T19:02:25.263 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.223892502Z level=info msg="Executing migration" id="Drop old annotation table v4" 2024-09-10T19:02:25.263 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.226488084Z level=info msg="Executing migration" id="create annotation table v5" 2024-09-10T19:02:25.264 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.229354387Z level=info msg="Executing migration" id="add index annotation 0 v3" 2024-09-10T19:02:25.264 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.231843418Z level=info msg="Executing migration" id="add index annotation 1 v3" 2024-09-10T19:02:25.264 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.234616658Z level=info msg="Executing migration" id="add index annotation 2 v3" 2024-09-10T19:02:25.264 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.237388334Z level=info msg="Executing migration" id="add index annotation 3 v3" 2024-09-10T19:02:25.264 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.23998439Z level=info msg="Executing migration" id="add index annotation 4 v3" 2024-09-10T19:02:25.264 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.240814727Z level=info msg="Executing migration" id="Update annotation table charset" 2024-09-10T19:02:25.264 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.241657871Z level=info msg="Executing migration" id="Add column region_id to annotation table" 2024-09-10T19:02:25.264 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.242546085Z level=info msg="Executing migration" id="Drop category_id index" 2024-09-10T19:02:25.264 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.243478954Z level=info msg="Executing migration" id="Add column tags to annotation table" 2024-09-10T19:02:25.264 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.244256411Z level=info msg="Executing migration" id="Create annotation_tag table v2" 2024-09-10T19:02:25.264 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.245224843Z level=info msg="Executing migration" id="Add unique index annotation_tag.annotation_id_tag_id" 2024-09-10T19:02:25.264 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.245929452Z level=info msg="Executing migration" id="drop index UQE_annotation_tag_annotation_id_tag_id - v2" 2024-09-10T19:02:25.264 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.246727473Z level=info msg="Executing migration" id="Rename table annotation_tag to annotation_tag_v2 - v2" 2024-09-10T19:02:25.264 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.248284306Z level=info msg="Executing migration" id="Create annotation_tag table v3" 2024-09-10T19:02:25.265 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.249152081Z level=info msg="Executing migration" id="create index UQE_annotation_tag_annotation_id_tag_id - Add unique index annotation_tag.annotation_id_tag_id V3" 2024-09-10T19:02:25.265 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.250005561Z level=info msg="Executing migration" id="copy annotation_tag v2 to v3" 2024-09-10T19:02:25.265 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.250941634Z level=info msg="Executing migration" id="drop table annotation_tag_v2" 2024-09-10T19:02:25.265 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.251906403Z level=info msg="Executing migration" id="Update alert annotations and set TEXT to empty" 2024-09-10T19:02:25.265 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.25250203Z level=info msg="Executing migration" id="Add created time to annotation table" 2024-09-10T19:02:25.265 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.253325066Z level=info msg="Executing migration" id="Add updated time to annotation table" 2024-09-10T19:02:25.265 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.254111236Z level=info msg="Executing migration" id="Add index for created in annotation table" 2024-09-10T19:02:25.265 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.254911014Z level=info msg="Executing migration" id="Add index for updated in annotation table" 2024-09-10T19:02:25.503 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.255695133Z level=info msg="Executing migration" id="Convert existing annotations from seconds to milliseconds" 2024-09-10T19:02:25.503 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.256538225Z level=info msg="Executing migration" id="Add epoch_end column" 2024-09-10T19:02:25.504 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.257420013Z level=info msg="Executing migration" id="Add index for epoch_end" 2024-09-10T19:02:25.504 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.258281845Z level=info msg="Executing migration" id="Make epoch_end the same as epoch" 2024-09-10T19:02:25.504 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.259122806Z level=info msg="Executing migration" id="Move region to single row" 2024-09-10T19:02:25.504 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.260011072Z level=info msg="Executing migration" id="Remove index org_id_epoch from annotation table" 2024-09-10T19:02:25.504 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.260871866Z level=info msg="Executing migration" id="Remove index org_id_dashboard_id_panel_id_epoch from annotation table" 2024-09-10T19:02:25.504 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.261730138Z level=info msg="Executing migration" id="Add index for org_id_dashboard_id_epoch_end_epoch on annotation table" 2024-09-10T19:02:25.504 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.262578272Z level=info msg="Executing migration" id="Add index for org_id_epoch_end_epoch on annotation table" 2024-09-10T19:02:25.504 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.2634643Z level=info msg="Executing migration" id="Remove index org_id_epoch_epoch_end from annotation table" 2024-09-10T19:02:25.504 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.264318706Z level=info msg="Executing migration" id="Add index for alert_id on annotation table" 2024-09-10T19:02:25.504 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.26517496Z level=info msg="Executing migration" id="Increase tags column to length 4096" 2024-09-10T19:02:25.504 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.266034208Z level=info msg="Executing migration" id="create test_data table" 2024-09-10T19:02:25.504 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.266443235Z level=info msg="Executing migration" id="create dashboard_version table v1" 2024-09-10T19:02:25.504 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.266893422Z level=info msg="Executing migration" id="add index dashboard_version.dashboard_id" 2024-09-10T19:02:25.505 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.267742989Z level=info msg="Executing migration" id="add unique index dashboard_version.dashboard_id and dashboard_version.version" 2024-09-10T19:02:25.505 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.268742046Z level=info msg="Executing migration" id="Set dashboard version to 1 where 0" 2024-09-10T19:02:25.505 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.269486408Z level=info msg="Executing migration" id="save existing dashboard data in dashboard_version table v1" 2024-09-10T19:02:25.505 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.270288851Z level=info msg="Executing migration" id="alter dashboard_version.data to mediumtext v1" 2024-09-10T19:02:25.506 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.270630163Z level=info msg="Executing migration" id="create team table" 2024-09-10T19:02:25.506 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.27319273Z level=info msg="Executing migration" id="add index team.org_id" 2024-09-10T19:02:25.506 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.275922928Z level=info msg="Executing migration" id="add unique index team_org_id_name" 2024-09-10T19:02:25.506 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.27870835Z level=info msg="Executing migration" id="create team member table" 2024-09-10T19:02:25.506 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.281534534Z level=info msg="Executing migration" id="add index team_member.org_id" 2024-09-10T19:02:25.506 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.284201426Z level=info msg="Executing migration" id="add unique index team_member_org_id_team_id_user_id" 2024-09-10T19:02:25.506 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.287076169Z level=info msg="Executing migration" id="add index team_member.team_id" 2024-09-10T19:02:25.506 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.289782773Z level=info msg="Executing migration" id="Add column email to team table" 2024-09-10T19:02:25.506 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.292420953Z level=info msg="Executing migration" id="Add column external to team_member table" 2024-09-10T19:02:25.506 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.295381643Z level=info msg="Executing migration" id="Add column permission to team_member table" 2024-09-10T19:02:25.506 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.298161092Z level=info msg="Executing migration" id="create dashboard acl table" 2024-09-10T19:02:25.506 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.300990196Z level=info msg="Executing migration" id="add index dashboard_acl_dashboard_id" 2024-09-10T19:02:25.506 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.303990855Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_user_id" 2024-09-10T19:02:25.507 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.306597191Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_team_id" 2024-09-10T19:02:25.507 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.307069609Z level=info msg="Executing migration" id="add index dashboard_acl_user_id" 2024-09-10T19:02:25.507 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.309880094Z level=info msg="Executing migration" id="add index dashboard_acl_team_id" 2024-09-10T19:02:25.507 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.312639921Z level=info msg="Executing migration" id="add index dashboard_acl_org_id_role" 2024-09-10T19:02:25.507 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.313209983Z level=info msg="Executing migration" id="add index dashboard_permission" 2024-09-10T19:02:25.507 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.313625752Z level=info msg="Executing migration" id="save default acl rules in dashboard_acl table" 2024-09-10T19:02:25.507 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.314097787Z level=info msg="Executing migration" id="delete acl rules for deleted dashboards and folders" 2024-09-10T19:02:25.507 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.314398279Z level=info msg="Executing migration" id="create tag table" 2024-09-10T19:02:25.507 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.314768752Z level=info msg="Executing migration" id="add index tag.key_value" 2024-09-10T19:02:25.507 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.315192033Z level=info msg="Executing migration" id="create login attempt table" 2024-09-10T19:02:25.507 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.315576193Z level=info msg="Executing migration" id="add index login_attempt.username" 2024-09-10T19:02:25.507 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.315996247Z level=info msg="Executing migration" id="drop index IDX_login_attempt_username - v1" 2024-09-10T19:02:25.507 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.316449566Z level=info msg="Executing migration" id="Rename table login_attempt to login_attempt_tmp_qwerty - v1" 2024-09-10T19:02:25.508 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.318455858Z level=info msg="Executing migration" id="create login_attempt v2" 2024-09-10T19:02:25.508 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.318841961Z level=info msg="Executing migration" id="create index IDX_login_attempt_username - v2" 2024-09-10T19:02:25.508 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.319268902Z level=info msg="Executing migration" id="copy login_attempt v1 to v2" 2024-09-10T19:02:25.508 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.319554583Z level=info msg="Executing migration" id="drop login_attempt_tmp_qwerty" 2024-09-10T19:02:25.508 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.319892642Z level=info msg="Executing migration" id="create user auth table" 2024-09-10T19:02:25.508 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.32026854Z level=info msg="Executing migration" id="create index IDX_user_auth_auth_module_auth_id - v1" 2024-09-10T19:02:25.508 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.320672815Z level=info msg="Executing migration" id="alter user_auth.auth_id to length 190" 2024-09-10T19:02:25.508 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.320934243Z level=info msg="Executing migration" id="Add OAuth access token to user_auth" 2024-09-10T19:02:25.508 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.321651671Z level=info msg="Executing migration" id="Add OAuth refresh token to user_auth" 2024-09-10T19:02:25.508 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.322400273Z level=info msg="Executing migration" id="Add OAuth token type to user_auth" 2024-09-10T19:02:25.508 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.323145734Z level=info msg="Executing migration" id="Add OAuth expiry to user_auth" 2024-09-10T19:02:25.508 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.323910797Z level=info msg="Executing migration" id="Add index to user_id column in user_auth" 2024-09-10T19:02:25.508 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.324329626Z level=info msg="Executing migration" id="Add OAuth ID token to user_auth" 2024-09-10T19:02:25.508 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.325084037Z level=info msg="Executing migration" id="create server_lock table" 2024-09-10T19:02:25.508 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.32546104Z level=info msg="Executing migration" id="add index server_lock.operation_uid" 2024-09-10T19:02:25.509 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.325887389Z level=info msg="Executing migration" id="create user auth token table" 2024-09-10T19:02:25.509 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.326304162Z level=info msg="Executing migration" id="add unique index user_auth_token.auth_token" 2024-09-10T19:02:25.509 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.329077365Z level=info msg="Executing migration" id="add unique index user_auth_token.prev_auth_token" 2024-09-10T19:02:25.509 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.32955691Z level=info msg="Executing migration" id="add index user_auth_token.user_id" 2024-09-10T19:02:25.509 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.330000052Z level=info msg="Executing migration" id="Add revoked_at to the user auth token" 2024-09-10T19:02:25.509 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.330794388Z level=info msg="Executing migration" id="create cache_data table" 2024-09-10T19:02:25.509 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.331242803Z level=info msg="Executing migration" id="add unique index cache_data.cache_key" 2024-09-10T19:02:25.509 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.33166768Z level=info msg="Executing migration" id="create short_url table v1" 2024-09-10T19:02:25.509 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.332134031Z level=info msg="Executing migration" id="add index short_url.org_id-uid" 2024-09-10T19:02:25.509 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.332560918Z level=info msg="Executing migration" id="alter table short_url alter column created_by type to bigint" 2024-09-10T19:02:25.509 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.332832797Z level=info msg="Executing migration" id="delete alert_definition table" 2024-09-10T19:02:25.509 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.333150091Z level=info msg="Executing migration" id="recreate alert_definition table" 2024-09-10T19:02:25.509 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.333582221Z level=info msg="Executing migration" id="add index in alert_definition on org_id and title columns" 2024-09-10T19:02:25.509 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.334016013Z level=info msg="Executing migration" id="add index in alert_definition on org_id and uid columns" 2024-09-10T19:02:25.510 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.334439325Z level=info msg="Executing migration" id="alter alert_definition table data column to mediumtext in mysql" 2024-09-10T19:02:25.510 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.334714437Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and title columns" 2024-09-10T19:02:25.510 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.33513283Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and uid columns" 2024-09-10T19:02:25.510 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.335548672Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and title columns" 2024-09-10T19:02:25.510 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.335968023Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and uid columns" 2024-09-10T19:02:25.510 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.336403261Z level=info msg="Executing migration" id="Add column paused in alert_definition" 2024-09-10T19:02:25.510 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.337201858Z level=info msg="Executing migration" id="drop alert_definition table" 2024-09-10T19:02:25.510 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.337645165Z level=info msg="Executing migration" id="delete alert_definition_version table" 2024-09-10T19:02:25.510 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.337930025Z level=info msg="Executing migration" id="recreate alert_definition_version table" 2024-09-10T19:02:25.510 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.338348047Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_id and version columns" 2024-09-10T19:02:25.510 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.3387495Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_uid and version columns" 2024-09-10T19:02:25.510 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.33918178Z level=info msg="Executing migration" id="alter alert_definition_version table data column to mediumtext in mysql" 2024-09-10T19:02:25.510 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.339492471Z level=info msg="Executing migration" id="drop alert_definition_version table" 2024-09-10T19:02:25.510 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.339876041Z level=info msg="Executing migration" id="create alert_instance table" 2024-09-10T19:02:25.510 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.340290015Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, def_uid and current_state columns" 2024-09-10T19:02:25.511 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.340684235Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, current_state columns" 2024-09-10T19:02:25.511 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.341103004Z level=info msg="Executing migration" id="add column current_state_end to alert_instance" 2024-09-10T19:02:25.511 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.341891046Z level=info msg="Executing migration" id="remove index def_org_id, def_uid, current_state on alert_instance" 2024-09-10T19:02:25.511 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.342302531Z level=info msg="Executing migration" id="remove index def_org_id, current_state on alert_instance" 2024-09-10T19:02:25.511 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.342691509Z level=info msg="Executing migration" id="rename def_org_id to rule_org_id in alert_instance" 2024-09-10T19:02:25.511 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.346187988Z level=info msg="Executing migration" id="rename def_uid to rule_uid in alert_instance" 2024-09-10T19:02:25.511 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.349784224Z level=info msg="Executing migration" id="add index rule_org_id, rule_uid, current_state on alert_instance" 2024-09-10T19:02:25.511 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.350292633Z level=info msg="Executing migration" id="add index rule_org_id, current_state on alert_instance" 2024-09-10T19:02:25.511 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.350688485Z level=info msg="Executing migration" id="add current_reason column related to current_state" 2024-09-10T19:02:25.511 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.351516067Z level=info msg="Executing migration" id="create alert_rule table" 2024-09-10T19:02:25.511 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.351968509Z level=info msg="Executing migration" id="add index in alert_rule on org_id and title columns" 2024-09-10T19:02:25.511 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.354764518Z level=info msg="Executing migration" id="add index in alert_rule on org_id and uid columns" 2024-09-10T19:02:25.511 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.357448601Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespace_uid, group_uid columns" 2024-09-10T19:02:25.511 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.358031891Z level=info msg="Executing migration" id="alter alert_rule table data column to mediumtext in mysql" 2024-09-10T19:02:25.512 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.360468631Z level=info msg="Executing migration" id="add column for to alert_rule" 2024-09-10T19:02:25.512 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.363032227Z level=info msg="Executing migration" id="add column annotations to alert_rule" 2024-09-10T19:02:25.512 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.365754789Z level=info msg="Executing migration" id="add column labels to alert_rule" 2024-09-10T19:02:25.512 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.366735148Z level=info msg="Executing migration" id="remove unique index from alert_rule on org_id, title columns" 2024-09-10T19:02:25.512 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.369259022Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespase_uid and title columns" 2024-09-10T19:02:25.512 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.371889611Z level=info msg="Executing migration" id="add dashboard_uid column to alert_rule" 2024-09-10T19:02:25.512 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.374494223Z level=info msg="Executing migration" id="add panel_id column to alert_rule" 2024-09-10T19:02:25.512 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.375423501Z level=info msg="Executing migration" id="add index in alert_rule on org_id, dashboard_uid and panel_id columns" 2024-09-10T19:02:25.513 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.37825797Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule" 2024-09-10T19:02:25.513 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.381068304Z level=info msg="Executing migration" id="add is_paused column to alert_rule table" 2024-09-10T19:02:25.513 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.38368786Z level=info msg="Executing migration" id="fix is_paused column for alert_rule table" 2024-09-10T19:02:25.513 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.384073924Z level=info msg="Executing migration" id="create alert_rule_version table" 2024-09-10T19:02:25.513 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.384952725Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_uid and version columns" 2024-09-10T19:02:25.513 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.385934936Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_namespace_uid and rule_group columns" 2024-09-10T19:02:25.513 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.388625569Z level=info msg="Executing migration" id="alter alert_rule_version table data column to mediumtext in mysql" 2024-09-10T19:02:25.513 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.388954252Z level=info msg="Executing migration" id="add column for to alert_rule_version" 2024-09-10T19:02:25.514 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.391800588Z level=info msg="Executing migration" id="add column annotations to alert_rule_version" 2024-09-10T19:02:25.514 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.394725518Z level=info msg="Executing migration" id="add column labels to alert_rule_version" 2024-09-10T19:02:25.514 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.39763Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule_version" 2024-09-10T19:02:25.514 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.398549924Z level=info msg="Executing migration" id="add is_paused column to alert_rule_versions table" 2024-09-10T19:02:25.514 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.401497628Z level=info msg="Executing migration" id="fix is_paused column for alert_rule_version table" 2024-09-10T19:02:25.514 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.404288225Z level=info msg="Executing migration" id=create_alert_configuration_table 2024-09-10T19:02:25.514 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.407093747Z level=info msg="Executing migration" id="Add column default in alert_configuration" 2024-09-10T19:02:25.514 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.407993423Z level=info msg="Executing migration" id="alert alert_configuration alertmanager_configuration column from TEXT to MEDIUMTEXT if mysql" 2024-09-10T19:02:25.514 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.410991292Z level=info msg="Executing migration" id="add column org_id in alert_configuration" 2024-09-10T19:02:25.514 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.413956204Z level=info msg="Executing migration" id="add index in alert_configuration table on org_id column" 2024-09-10T19:02:25.514 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.416928919Z level=info msg="Executing migration" id="add configuration_hash column to alert_configuration" 2024-09-10T19:02:25.514 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.417879094Z level=info msg="Executing migration" id=create_ngalert_configuration_table 2024-09-10T19:02:25.514 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.42072451Z level=info msg="Executing migration" id="add index in ngalert_configuration on org_id column" 2024-09-10T19:02:25.514 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.423617428Z level=info msg="Executing migration" id="add column send_alerts_to in ngalert_configuration" 2024-09-10T19:02:25.515 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.426561954Z level=info msg="Executing migration" id="create provenance_type table" 2024-09-10T19:02:25.515 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.429486248Z level=info msg="Executing migration" id="add index to uniquify (record_key, record_type, org_id) columns" 2024-09-10T19:02:25.515 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.432183224Z level=info msg="Executing migration" id="create alert_image table" 2024-09-10T19:02:25.515 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.434855759Z level=info msg="Executing migration" id="add unique index on token to alert_image table" 2024-09-10T19:02:25.515 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.437341378Z level=info msg="Executing migration" id="support longer URLs in alert_image table" 2024-09-10T19:02:25.515 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.440193487Z level=info msg="Executing migration" id=create_alert_configuration_history_table 2024-09-10T19:02:25.515 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.442886939Z level=info msg="Executing migration" id="drop non-unique orgID index on alert_configuration" 2024-09-10T19:02:25.515 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.445389851Z level=info msg="Executing migration" id="drop unique orgID index on alert_configuration if exists" 2024-09-10T19:02:25.515 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.44560313Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop unique orgID index on alert_configuration if exists" 2024-09-10T19:02:25.515 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.447997172Z level=info msg="Executing migration" id="extract alertmanager configuration history to separate table" 2024-09-10T19:02:25.515 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.450813877Z level=info msg="Executing migration" id="add unique index on orgID to alert_configuration" 2024-09-10T19:02:25.515 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.453274981Z level=info msg="Executing migration" id="clear migration entry \"remove unified alerting data\"" 2024-09-10T19:02:25.515 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.45342361Z level=info msg="Executing migration" id="move dashboard alerts to unified alerting" 2024-09-10T19:02:25.515 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.453573945Z level=info msg="alerts found to migrate" alerts=0 2024-09-10T19:02:25.516 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.456007771Z level=info msg="Executing migration" id="create library_element table v1" 2024-09-10T19:02:25.516 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.458675614Z level=info msg="Executing migration" id="add index library_element org_id-folder_id-name-kind" 2024-09-10T19:02:25.516 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.461369902Z level=info msg="Executing migration" id="create library_element_connection table v1" 2024-09-10T19:02:25.516 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.464106392Z level=info msg="Executing migration" id="add index library_element_connection element_id-kind-connection_id" 2024-09-10T19:02:25.516 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.466597201Z level=info msg="Executing migration" id="add unique index library_element org_id_uid" 2024-09-10T19:02:25.516 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.4691146Z level=info msg="Executing migration" id="increase max description length to 2048" 2024-09-10T19:02:25.516 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.471921719Z level=info msg="Executing migration" id="clone move dashboard alerts to unified alerting" 2024-09-10T19:02:25.516 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.474557904Z level=info msg="Executing migration" id="create data_keys table" 2024-09-10T19:02:25.516 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.475238078Z level=info msg="Executing migration" id="create secrets table" 2024-09-10T19:02:25.516 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.476182444Z level=info msg="Executing migration" id="rename data_keys name column to id" 2024-09-10T19:02:25.516 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.480662738Z level=info msg="Executing migration" id="add name column into data_keys" 2024-09-10T19:02:25.516 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.48157559Z level=info msg="Executing migration" id="copy data_keys id column values into name" 2024-09-10T19:02:25.516 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.482541152Z level=info msg="Executing migration" id="rename data_keys name column to label" 2024-09-10T19:02:25.516 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.486859164Z level=info msg="Executing migration" id="rename data_keys id column back to name" 2024-09-10T19:02:25.517 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.491158674Z level=info msg="Executing migration" id="create kv_store table v1" 2024-09-10T19:02:25.517 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.49200976Z level=info msg="Executing migration" id="add index kv_store.org_id-namespace-key" 2024-09-10T19:02:25.517 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.492862561Z level=info msg="Executing migration" id="update dashboard_uid and panel_id from existing annotations" 2024-09-10T19:02:25.517 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.493711276Z level=info msg="Executing migration" id="create permission table" 2024-09-10T19:02:25.517 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.494699387Z level=info msg="Executing migration" id="add unique index permission.role_id" 2024-09-10T19:02:25.517 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.49543814Z level=info msg="Executing migration" id="add unique index role_id_action_scope" 2024-09-10T19:02:25.517 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.496296036Z level=info msg="Executing migration" id="create role table" 2024-09-10T19:02:25.517 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.49715626Z level=info msg="Executing migration" id="add column display_name" 2024-09-10T19:02:25.517 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.498120539Z level=info msg="Executing migration" id="add column group_name" 2024-09-10T19:02:25.517 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.499044646Z level=info msg="Executing migration" id="add index role.org_id" 2024-09-10T19:02:25.517 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.499924305Z level=info msg="Executing migration" id="add unique index role_org_id_name" 2024-09-10T19:02:25.517 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.500893815Z level=info msg="Executing migration" id="add index role_org_id_uid" 2024-09-10T19:02:25.517 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.501616971Z level=info msg="Executing migration" id="create team role table" 2024-09-10T19:02:25.517 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.502624368Z level=info msg="Executing migration" id="add index team_role.org_id" 2024-09-10T19:02:25.518 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.503368764Z level=info msg="Executing migration" id="add unique index team_role_org_id_team_id_role_id" 2024-09-10T19:02:25.518 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.504257652Z level=info msg="Executing migration" id="add index team_role.team_id" 2024-09-10T19:02:25.518 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.505121409Z level=info msg="Executing migration" id="create user role table" 2024-09-10T19:02:25.518 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.505963338Z level=info msg="Executing migration" id="add index user_role.org_id" 2024-09-10T19:02:25.518 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.506824163Z level=info msg="Executing migration" id="add unique index user_role_org_id_user_id_role_id" 2024-09-10T19:02:25.518 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.50766822Z level=info msg="Executing migration" id="add index user_role.user_id" 2024-09-10T19:02:25.759 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.508529951Z level=info msg="Executing migration" id="create builtin role table" 2024-09-10T19:02:25.759 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.509377232Z level=info msg="Executing migration" id="add index builtin_role.role_id" 2024-09-10T19:02:25.759 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.510243998Z level=info msg="Executing migration" id="add index builtin_role.name" 2024-09-10T19:02:25.759 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.511122365Z level=info msg="Executing migration" id="Add column org_id to builtin_role table" 2024-09-10T19:02:25.760 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.512127989Z level=info msg="Executing migration" id="add index builtin_role.org_id" 2024-09-10T19:02:25.760 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.512962766Z level=info msg="Executing migration" id="add unique index builtin_role_org_id_role_id_role" 2024-09-10T19:02:25.760 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.513837123Z level=info msg="Executing migration" id="Remove unique index role_org_id_uid" 2024-09-10T19:02:25.760 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.51469016Z level=info msg="Executing migration" id="add unique index role.uid" 2024-09-10T19:02:25.760 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.515577226Z level=info msg="Executing migration" id="create seed assignment table" 2024-09-10T19:02:25.760 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.518591122Z level=info msg="Executing migration" id="add unique index builtin_role_role_name" 2024-09-10T19:02:25.760 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.521467927Z level=info msg="Executing migration" id="add column hidden to role table" 2024-09-10T19:02:25.760 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.52426473Z level=info msg="Executing migration" id="create query_history table v1" 2024-09-10T19:02:25.760 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.527076536Z level=info msg="Executing migration" id="add index query_history.org_id-created_by-datasource_uid" 2024-09-10T19:02:25.760 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.530088015Z level=info msg="Executing migration" id="alter table query_history alter column created_by type to bigint" 2024-09-10T19:02:25.760 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.532924505Z level=info msg="Executing migration" id="teams permissions migration" 2024-09-10T19:02:25.760 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.535681865Z level=info msg="Executing migration" id="dashboard permissions" 2024-09-10T19:02:25.760 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.538597483Z level=info msg="Executing migration" id="dashboard permissions uid scopes" 2024-09-10T19:02:25.761 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.541400552Z level=info msg="Executing migration" id="drop managed folder create actions" 2024-09-10T19:02:25.761 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.54413684Z level=info msg="Executing migration" id="alerting notification permissions" 2024-09-10T19:02:25.761 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.546906818Z level=info msg="Executing migration" id="create query_history_star table v1" 2024-09-10T19:02:25.761 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.549806705Z level=info msg="Executing migration" id="add index query_history.user_id-query_uid" 2024-09-10T19:02:25.761 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.552785544Z level=info msg="Executing migration" id="add column org_id in query_history_star" 2024-09-10T19:02:25.761 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.555343815Z level=info msg="Executing migration" id="alter table query_history_star_mig column user_id type to bigint" 2024-09-10T19:02:25.761 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.558195079Z level=info msg="Executing migration" id="create correlation table v1" 2024-09-10T19:02:25.761 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.561220649Z level=info msg="Executing migration" id="add index correlations.uid" 2024-09-10T19:02:25.761 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.56405132Z level=info msg="Executing migration" id="add index correlations.source_uid" 2024-09-10T19:02:25.761 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.566688228Z level=info msg="Executing migration" id="add correlation config column" 2024-09-10T19:02:25.761 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.569400899Z level=info msg="Executing migration" id="create entity_events table" 2024-09-10T19:02:25.761 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.572143624Z level=info msg="Executing migration" id="create dashboard public config v1" 2024-09-10T19:02:25.761 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.574739844Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v1" 2024-09-10T19:02:25.761 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.574900038Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index UQE_dashboard_public_config_uid - v1" 2024-09-10T19:02:25.762 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.577404965Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-10T19:02:25.762 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.577499479Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-10T19:02:25.762 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.580104123Z level=info msg="Executing migration" id="Drop old dashboard public config table" 2024-09-10T19:02:25.762 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.583097515Z level=info msg="Executing migration" id="recreate dashboard public config v1" 2024-09-10T19:02:25.762 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.585459028Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v1" 2024-09-10T19:02:25.762 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.588231448Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-10T19:02:25.762 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.590914114Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v2" 2024-09-10T19:02:25.763 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.593567864Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-09-10T19:02:25.763 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.596175485Z level=info msg="Executing migration" id="Drop public config table" 2024-09-10T19:02:25.763 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.599052619Z level=info msg="Executing migration" id="Recreate dashboard public config v2" 2024-09-10T19:02:25.763 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.601613343Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v2" 2024-09-10T19:02:25.763 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.604279509Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-09-10T19:02:25.763 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.607090614Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_access_token - v2" 2024-09-10T19:02:25.763 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.609920044Z level=info msg="Executing migration" id="Rename table dashboard_public_config to dashboard_public - v2" 2024-09-10T19:02:25.763 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.613040621Z level=info msg="Executing migration" id="add annotations_enabled column" 2024-09-10T19:02:25.763 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.614083931Z level=info msg="Executing migration" id="add time_selection_enabled column" 2024-09-10T19:02:25.763 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.615109843Z level=info msg="Executing migration" id="delete orphaned public dashboards" 2024-09-10T19:02:25.763 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.615970293Z level=info msg="Executing migration" id="add share column" 2024-09-10T19:02:25.763 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.616967542Z level=info msg="Executing migration" id="create default alerting folders" 2024-09-10T19:02:25.764 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.617816329Z level=info msg="Executing migration" id="create file table" 2024-09-10T19:02:25.764 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.6187182Z level=info msg="Executing migration" id="file table idx: path natural pk" 2024-09-10T19:02:25.764 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.619606392Z level=info msg="Executing migration" id="file table idx: parent_folder_path_hash fast folder retrieval" 2024-09-10T19:02:25.764 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.620378561Z level=info msg="Executing migration" id="create file_meta table" 2024-09-10T19:02:25.764 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.621307046Z level=info msg="Executing migration" id="file table idx: path key" 2024-09-10T19:02:25.764 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.622045857Z level=info msg="Executing migration" id="set path collation in file table" 2024-09-10T19:02:25.764 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.623024456Z level=info msg="Executing migration" id="managed permissions migration" 2024-09-10T19:02:25.764 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.623803951Z level=info msg="Executing migration" id="managed folder permissions alert actions migration" 2024-09-10T19:02:25.764 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.624634658Z level=info msg="Executing migration" id="RBAC action name migrator" 2024-09-10T19:02:25.764 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.625457412Z level=info msg="Executing migration" id="Add UID column to playlist" 2024-09-10T19:02:25.764 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.626524112Z level=info msg="Executing migration" id="Update uid column values in playlist" 2024-09-10T19:02:25.764 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.627519674Z level=info msg="Executing migration" id="Add index for uid in playlist" 2024-09-10T19:02:25.764 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.628239689Z level=info msg="Executing migration" id="update group index for alert rules" 2024-09-10T19:02:25.764 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.629184015Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated migration" 2024-09-10T19:02:25.765 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.629931457Z level=info msg="Executing migration" id="admin only folder/dashboard permission" 2024-09-10T19:02:25.765 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.631011307Z level=info msg="Executing migration" id="add action column to seed_assignment" 2024-09-10T19:02:25.765 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.632125588Z level=info msg="Executing migration" id="add scope column to seed_assignment" 2024-09-10T19:02:25.765 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.633132947Z level=info msg="Executing migration" id="remove unique index builtin_role_role_name before nullable update" 2024-09-10T19:02:25.765 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.634025505Z level=info msg="Executing migration" id="update seed_assignment role_name column to nullable" 2024-09-10T19:02:25.765 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.643732465Z level=info msg="Executing migration" id="add unique index builtin_role_name back" 2024-09-10T19:02:25.765 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.646720206Z level=info msg="Executing migration" id="add unique index builtin_role_action_scope" 2024-09-10T19:02:25.765 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.649396234Z level=info msg="Executing migration" id="add primary key to seed_assigment" 2024-09-10T19:02:25.765 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.652788792Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated fixed migration" 2024-09-10T19:02:25.765 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.655666482Z level=info msg="Executing migration" id="migrate external alertmanagers to datsourcse" 2024-09-10T19:02:25.765 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.658475579Z level=info msg="Executing migration" id="create folder table" 2024-09-10T19:02:25.765 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.661167831Z level=info msg="Executing migration" id="Add index for parent_uid" 2024-09-10T19:02:25.765 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.663999745Z level=info msg="Executing migration" id="Add unique index for folder.uid and folder.org_id" 2024-09-10T19:02:25.765 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.666626692Z level=info msg="Executing migration" id="Update folder title length" 2024-09-10T19:02:25.766 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.669524809Z level=info msg="Executing migration" id="Add unique index for folder.title and folder.parent_uid" 2024-09-10T19:02:25.766 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=migrator t=2024-09-10T19:02:25.67213758Z level=info msg="migrations completed" performed=485 skipped=0 duration=758.309183ms 2024-09-10T19:02:25.766 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=sqlstore t=2024-09-10T19:02:25.672556542Z level=info msg="Created default organization" 2024-09-10T19:02:25.766 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=plugin.loader t=2024-09-10T19:02:25.731943307Z level=info msg="Plugin registered" pluginID=input 2024-09-10T19:02:25.766 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=plugin.loader t=2024-09-10T19:02:25.758516672Z level=info msg="Plugin registered" pluginID=grafana-piechart-panel 2024-09-10T19:02:25.766 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=plugin.loader t=2024-09-10T19:02:25.758535196Z level=info msg="Plugin registered" pluginID=vonage-status-panel 2024-09-10T19:02:25.766 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=secrets t=2024-09-10T19:02:25.758567248Z level=info msg="Envelope encryption state" enabled=true currentprovider=secretKey.v1 2024-09-10T19:02:25.893 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 10 19:02:25 smithi047 bash[72910]: Trying to pull quay.io/prometheus/node-exporter:v1.5.0... 2024-09-10T19:02:26.093 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=query_data t=2024-09-10T19:02:25.760631781Z level=info msg="Query Service initialization" 2024-09-10T19:02:26.093 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=live.push_http t=2024-09-10T19:02:25.763177798Z level=info msg="Live Push Gateway initialization" 2024-09-10T19:02:26.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:26.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:26.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:26.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:26.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-mon[33230]: Deploying daemon node-exporter.a on smithi047 2024-09-10T19:02:26.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:25 smithi149 ceph-mon[33230]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:02:26.182 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:25 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:26.182 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:25 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:26.183 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:25 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:26.183 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:25 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:26.183 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:25 smithi047 ceph-mon[39856]: Deploying daemon node-exporter.a on smithi047 2024-09-10T19:02:26.183 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:25 smithi047 ceph-mon[39856]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:02:26.183 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:25 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:26.183 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:25 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:26.183 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:25 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:26.183 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:25 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:26.183 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:25 smithi047 ceph-mon[29019]: Deploying daemon node-exporter.a on smithi047 2024-09-10T19:02:26.183 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:25 smithi047 ceph-mon[29019]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:02:26.539 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 10 19:02:26 smithi047 bash[72910]: Getting image source signatures 2024-09-10T19:02:26.539 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 10 19:02:26 smithi047 bash[72910]: Copying blob sha256:c0e87333d3805a7b7e1c25783dd25f27da0fe70a8e262eee7ed190c8dc843fa7 2024-09-10T19:02:26.539 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 10 19:02:26 smithi047 bash[72910]: Copying blob sha256:5c12815fee558b157dee7f7509dedbaba0a8379098858a65ec869e1f1526ea0c 2024-09-10T19:02:26.539 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 10 19:02:26 smithi047 bash[72910]: Copying blob sha256:22b70bddd3acadc892fca4c2af4260629bfda5dfd11ebc106a93ce24e752b5ed 2024-09-10T19:02:26.724 INFO:teuthology.orchestra.run.smithi047.stderr:++ ceph auth get-key osd.0 2024-09-10T19:02:26.843 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:26 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=infra.usagestats.collector t=2024-09-10T19:02:26.500513287Z level=info msg="registering usage stat providers" usageStatsProvidersLen=2 2024-09-10T19:02:26.843 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:26 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=provisioning.datasources t=2024-09-10T19:02:26.50081375Z level=warn msg="[Deprecated] the datasource provisioning config is outdated. please upgrade" filename=/etc/grafana/provisioning/datasources/ceph-dashboard.yml 2024-09-10T19:02:26.843 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:26 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=provisioning.datasources t=2024-09-10T19:02:26.53324367Z level=info msg="inserting datasource from configuration " name=Dashboard1 uid=P43CA22E17D0F9596 2024-09-10T19:02:26.843 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:26 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=provisioning.datasources t=2024-09-10T19:02:26.565586291Z level=info msg="inserting datasource from configuration " name=Loki uid=P8E80F9AEF21F6940 2024-09-10T19:02:26.843 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:26 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=provisioning.alerting t=2024-09-10T19:02:26.587068383Z level=info msg="starting to provision alerting" 2024-09-10T19:02:26.843 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:26 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=provisioning.alerting t=2024-09-10T19:02:26.587089081Z level=info msg="finished to provision alerting" 2024-09-10T19:02:26.843 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:26 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=ngalert.state.manager t=2024-09-10T19:02:26.587184558Z level=info msg="Warming state cache for startup" 2024-09-10T19:02:26.843 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:26 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=grafanaStorageLogger t=2024-09-10T19:02:26.587220836Z level=info msg="storage starting" 2024-09-10T19:02:26.844 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:26 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=http.server t=2024-09-10T19:02:26.588856935Z level=info msg="HTTP Server Listen" address=[::]:3000 protocol=https subUrl= socket= 2024-09-10T19:02:26.844 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:26 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=sqlstore.transactions t=2024-09-10T19:02:26.597841345Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-10T19:02:26.844 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:26 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=ngalert.state.manager t=2024-09-10T19:02:26.597916206Z level=info msg="State cache has been initialized" states=0 duration=10.730507ms 2024-09-10T19:02:26.844 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:26 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=ticker t=2024-09-10T19:02:26.597974511Z level=info msg=starting first_tick=2024-09-10T19:02:30Z 2024-09-10T19:02:26.844 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:26 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=ngalert.multiorg.alertmanager t=2024-09-10T19:02:26.597972269Z level=info msg="starting MultiOrg Alertmanager" 2024-09-10T19:02:26.844 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:26 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=sqlstore.transactions t=2024-09-10T19:02:26.64246003Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-10T19:02:26.844 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:26 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=sqlstore.transactions t=2024-09-10T19:02:26.674217933Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-10T19:02:26.844 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:02:26 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a[64005]: logger=sqlstore.transactions t=2024-09-10T19:02:26.684593184Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=1 code="database is locked" 2024-09-10T19:02:27.045 INFO:teuthology.orchestra.run.smithi047.stderr:+ NK=AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== 2024-09-10T19:02:27.045 INFO:teuthology.orchestra.run.smithi047.stderr:+ '[' AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== == AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== ']' 2024-09-10T19:02:27.045 INFO:teuthology.orchestra.run.smithi047.stderr:+ sleep 5 2024-09-10T19:02:27.165 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:27 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [7, 2]}]: dispatch 2024-09-10T19:02:27.165 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:27 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2024-09-10T19:02:27.165 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:27 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2024-09-10T19:02:27.165 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:27 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2024-09-10T19:02:27.165 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:27 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/3356352328' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:02:27.166 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:27 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:02:27.166 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:27 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:27.539 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:27 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [7, 2]}]: dispatch 2024-09-10T19:02:27.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:27 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2024-09-10T19:02:27.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:27 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2024-09-10T19:02:27.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:27 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2024-09-10T19:02:27.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:27 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/3356352328' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:02:27.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:27 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:02:27.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:27 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:27.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:27 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [7, 2]}]: dispatch 2024-09-10T19:02:27.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:27 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2024-09-10T19:02:27.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:27 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2024-09-10T19:02:27.541 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:27 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2024-09-10T19:02:27.541 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:27 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/3356352328' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:02:27.541 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:27 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:02:27.541 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:27 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:28.039 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 10 19:02:27 smithi047 bash[72910]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-09-10T19:02:28.040 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 10 19:02:27 smithi047 bash[72910]: Writing manifest to image destination 2024-09-10T19:02:28.343 INFO:journalctl@ceph.iscsi.iscsi.a.smithi149.stdout:Sep 10 19:02:28 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-iscsi-iscsi-a[60538]: debug there is no tcmu-runner data available 2024-09-10T19:02:28.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:28 smithi149 ceph-mon[33230]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:02:28.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:28 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [7, 2]}]': finished 2024-09-10T19:02:28.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:28 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]': finished 2024-09-10T19:02:28.344 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:28 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]': finished 2024-09-10T19:02:28.344 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:28 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]': finished 2024-09-10T19:02:28.344 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:28 smithi149 ceph-mon[33230]: osdmap e63: 8 total, 8 up, 8 in 2024-09-10T19:02:28.344 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:28 smithi149 ceph-mon[33230]: osdmap e64: 8 total, 8 up, 8 in 2024-09-10T19:02:28.393 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 10 19:02:28 smithi047 podman[72910]: 2024-09-10 19:02:28.077836882 +0000 UTC m=+2.353682360 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-09-10T19:02:28.394 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 10 19:02:28 smithi047 podman[72910]: 2024-09-10 19:02:28.191165163 +0000 UTC m=+2.467010623 container create 39204f3adcb67a79738d76e4a7e7a99d78efce4167a277308478638a295ab599 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-10T19:02:28.394 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:28 smithi047 ceph-mon[29019]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:02:28.394 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:28 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [7, 2]}]': finished 2024-09-10T19:02:28.394 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:28 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]': finished 2024-09-10T19:02:28.394 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:28 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]': finished 2024-09-10T19:02:28.394 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:28 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]': finished 2024-09-10T19:02:28.394 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:28 smithi047 ceph-mon[29019]: osdmap e63: 8 total, 8 up, 8 in 2024-09-10T19:02:28.394 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:28 smithi047 ceph-mon[29019]: osdmap e64: 8 total, 8 up, 8 in 2024-09-10T19:02:28.394 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:28 smithi047 ceph-mon[39856]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:02:28.395 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:28 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [7, 2]}]': finished 2024-09-10T19:02:28.395 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:28 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]': finished 2024-09-10T19:02:28.395 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:28 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]': finished 2024-09-10T19:02:28.395 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:28 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]': finished 2024-09-10T19:02:28.395 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:28 smithi047 ceph-mon[39856]: osdmap e63: 8 total, 8 up, 8 in 2024-09-10T19:02:28.395 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:28 smithi047 ceph-mon[39856]: osdmap e64: 8 total, 8 up, 8 in 2024-09-10T19:02:28.790 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 10 19:02:28 smithi047 podman[72910]: 2024-09-10 19:02:28.393288411 +0000 UTC m=+2.669133861 container init 39204f3adcb67a79738d76e4a7e7a99d78efce4167a277308478638a295ab599 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-10T19:02:28.790 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 10 19:02:28 smithi047 podman[72910]: 2024-09-10 19:02:28.39728488 +0000 UTC m=+2.673130337 container start 39204f3adcb67a79738d76e4a7e7a99d78efce4167a277308478638a295ab599 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-10T19:02:28.790 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 10 19:02:28 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-a[73008]: ts=2024-09-10T19:02:28.401Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-09-10T19:02:28.790 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 10 19:02:28 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-a[73008]: ts=2024-09-10T19:02:28.401Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-09-10T19:02:28.790 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 10 19:02:28 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-a[73008]: ts=2024-09-10T19:02:28.402Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-09-10T19:02:28.790 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 10 19:02:28 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-a[73008]: ts=2024-09-10T19:02:28.402Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-09-10T19:02:28.791 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 10 19:02:28 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-a[73008]: ts=2024-09-10T19:02:28.403Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-09-10T19:02:28.791 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 10 19:02:28 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-a[73008]: ts=2024-09-10T19:02:28.403Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-09-10T19:02:28.791 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 10 19:02:28 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-a[73008]: ts=2024-09-10T19:02:28.403Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-09-10T19:02:28.791 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 10 19:02:28 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-a[73008]: ts=2024-09-10T19:02:28.403Z caller=node_exporter.go:117 level=info collector=arp 2024-09-10T19:02:28.791 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 10 19:02:28 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-a[73008]: ts=2024-09-10T19:02:28.403Z caller=node_exporter.go:117 level=info collector=bcache 2024-09-10T19:02:28.791 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 10 19:02:28 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-a[73008]: ts=2024-09-10T19:02:28.403Z caller=node_exporter.go:117 level=info collector=bonding 2024-09-10T19:02:28.791 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 10 19:02:28 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-a[73008]: ts=2024-09-10T19:02:28.403Z caller=node_exporter.go:117 level=info collector=btrfs 2024-09-10T19:02:28.791 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 10 19:02:28 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-a[73008]: ts=2024-09-10T19:02:28.403Z caller=node_exporter.go:117 level=info collector=conntrack 2024-09-10T19:02:28.791 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 10 19:02:28 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-a[73008]: ts=2024-09-10T19:02:28.403Z caller=node_exporter.go:117 level=info collector=cpu 2024-09-10T19:02:28.791 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 10 19:02:28 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-a[73008]: ts=2024-09-10T19:02:28.403Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-09-10T19:02:28.792 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 10 19:02:28 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-a[73008]: ts=2024-09-10T19:02:28.403Z caller=node_exporter.go:117 level=info collector=diskstats 2024-09-10T19:02:28.792 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 10 19:02:28 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-a[73008]: ts=2024-09-10T19:02:28.403Z caller=node_exporter.go:117 level=info collector=dmi 2024-09-10T19:02:28.792 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 10 19:02:28 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-a[73008]: ts=2024-09-10T19:02:28.403Z caller=node_exporter.go:117 level=info collector=edac 2024-09-10T19:02:28.792 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 10 19:02:28 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-a[73008]: ts=2024-09-10T19:02:28.403Z caller=node_exporter.go:117 level=info collector=entropy 2024-09-10T19:02:28.792 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 10 19:02:28 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-a[73008]: ts=2024-09-10T19:02:28.403Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-09-10T19:02:28.792 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 10 19:02:28 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-a[73008]: ts=2024-09-10T19:02:28.403Z caller=node_exporter.go:117 level=info collector=filefd 2024-09-10T19:02:28.792 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 10 19:02:28 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-a[73008]: ts=2024-09-10T19:02:28.403Z caller=node_exporter.go:117 level=info collector=filesystem 2024-09-10T19:02:28.792 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 10 19:02:28 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-a[73008]: ts=2024-09-10T19:02:28.403Z caller=node_exporter.go:117 level=info collector=hwmon 2024-09-10T19:02:28.793 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 10 19:02:28 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-a[73008]: ts=2024-09-10T19:02:28.403Z caller=node_exporter.go:117 level=info collector=infiniband 2024-09-10T19:02:28.793 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 10 19:02:28 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-a[73008]: ts=2024-09-10T19:02:28.403Z caller=node_exporter.go:117 level=info collector=ipvs 2024-09-10T19:02:28.793 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 10 19:02:28 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-a[73008]: ts=2024-09-10T19:02:28.403Z caller=node_exporter.go:117 level=info collector=loadavg 2024-09-10T19:02:28.793 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 10 19:02:28 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-a[73008]: ts=2024-09-10T19:02:28.403Z caller=node_exporter.go:117 level=info collector=mdadm 2024-09-10T19:02:28.793 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 10 19:02:28 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-a[73008]: ts=2024-09-10T19:02:28.403Z caller=node_exporter.go:117 level=info collector=meminfo 2024-09-10T19:02:28.793 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 10 19:02:28 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-a[73008]: ts=2024-09-10T19:02:28.403Z caller=node_exporter.go:117 level=info collector=netclass 2024-09-10T19:02:28.793 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 10 19:02:28 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-a[73008]: ts=2024-09-10T19:02:28.403Z caller=node_exporter.go:117 level=info collector=netdev 2024-09-10T19:02:28.793 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 10 19:02:28 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-a[73008]: ts=2024-09-10T19:02:28.403Z caller=node_exporter.go:117 level=info collector=netstat 2024-09-10T19:02:28.793 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 10 19:02:28 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-a[73008]: ts=2024-09-10T19:02:28.403Z caller=node_exporter.go:117 level=info collector=nfs 2024-09-10T19:02:28.793 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 10 19:02:28 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-a[73008]: ts=2024-09-10T19:02:28.403Z caller=node_exporter.go:117 level=info collector=nfsd 2024-09-10T19:02:28.793 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 10 19:02:28 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-a[73008]: ts=2024-09-10T19:02:28.403Z caller=node_exporter.go:117 level=info collector=nvme 2024-09-10T19:02:28.793 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 10 19:02:28 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-a[73008]: ts=2024-09-10T19:02:28.403Z caller=node_exporter.go:117 level=info collector=os 2024-09-10T19:02:28.793 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 10 19:02:28 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-a[73008]: ts=2024-09-10T19:02:28.403Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-09-10T19:02:28.794 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 10 19:02:28 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-a[73008]: ts=2024-09-10T19:02:28.403Z caller=node_exporter.go:117 level=info collector=pressure 2024-09-10T19:02:28.794 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 10 19:02:28 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-a[73008]: ts=2024-09-10T19:02:28.403Z caller=node_exporter.go:117 level=info collector=rapl 2024-09-10T19:02:28.794 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 10 19:02:28 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-a[73008]: ts=2024-09-10T19:02:28.403Z caller=node_exporter.go:117 level=info collector=schedstat 2024-09-10T19:02:28.794 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 10 19:02:28 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-a[73008]: ts=2024-09-10T19:02:28.403Z caller=node_exporter.go:117 level=info collector=selinux 2024-09-10T19:02:28.794 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 10 19:02:28 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-a[73008]: ts=2024-09-10T19:02:28.403Z caller=node_exporter.go:117 level=info collector=sockstat 2024-09-10T19:02:28.794 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 10 19:02:28 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-a[73008]: ts=2024-09-10T19:02:28.403Z caller=node_exporter.go:117 level=info collector=softnet 2024-09-10T19:02:28.794 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 10 19:02:28 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-a[73008]: ts=2024-09-10T19:02:28.403Z caller=node_exporter.go:117 level=info collector=stat 2024-09-10T19:02:28.794 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 10 19:02:28 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-a[73008]: ts=2024-09-10T19:02:28.403Z caller=node_exporter.go:117 level=info collector=tapestats 2024-09-10T19:02:28.794 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 10 19:02:28 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-a[73008]: ts=2024-09-10T19:02:28.403Z caller=node_exporter.go:117 level=info collector=textfile 2024-09-10T19:02:28.794 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 10 19:02:28 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-a[73008]: ts=2024-09-10T19:02:28.403Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-09-10T19:02:28.794 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 10 19:02:28 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-a[73008]: ts=2024-09-10T19:02:28.403Z caller=node_exporter.go:117 level=info collector=time 2024-09-10T19:02:28.794 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 10 19:02:28 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-a[73008]: ts=2024-09-10T19:02:28.403Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-09-10T19:02:28.795 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 10 19:02:28 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-a[73008]: ts=2024-09-10T19:02:28.403Z caller=node_exporter.go:117 level=info collector=uname 2024-09-10T19:02:28.795 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 10 19:02:28 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-a[73008]: ts=2024-09-10T19:02:28.403Z caller=node_exporter.go:117 level=info collector=vmstat 2024-09-10T19:02:28.795 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 10 19:02:28 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-a[73008]: ts=2024-09-10T19:02:28.403Z caller=node_exporter.go:117 level=info collector=xfs 2024-09-10T19:02:28.795 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 10 19:02:28 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-a[73008]: ts=2024-09-10T19:02:28.403Z caller=node_exporter.go:117 level=info collector=zfs 2024-09-10T19:02:28.795 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 10 19:02:28 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-a[73008]: ts=2024-09-10T19:02:28.404Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-09-10T19:02:28.795 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 10 19:02:28 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-a[73008]: ts=2024-09-10T19:02:28.404Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-09-10T19:02:28.795 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 10 19:02:28 smithi047 bash[72910]: 39204f3adcb67a79738d76e4a7e7a99d78efce4167a277308478638a295ab599 2024-09-10T19:02:28.795 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 10 19:02:28 smithi047 systemd[1]: Started Ceph node-exporter.a for d8563d7e-6fa5-11ef-bcea-c7b262605968. 2024-09-10T19:02:29.275 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:29 smithi149 ceph-mon[33230]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:02:29.276 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:29 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:29.276 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:29 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:29.276 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:29 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:29.276 INFO:journalctl@ceph.node-exporter.b.smithi149.stdout:Sep 10 19:02:29 smithi149 systemd[1]: Starting Ceph node-exporter.b for d8563d7e-6fa5-11ef-bcea-c7b262605968... 2024-09-10T19:02:29.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:29 smithi047 ceph-mon[29019]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:02:29.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:29 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:29.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:29 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:29.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:29 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:29.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:29 smithi047 ceph-mon[39856]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:02:29.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:29 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:29.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:29 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:29.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:29 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:29.593 INFO:journalctl@ceph.node-exporter.b.smithi149.stdout:Sep 10 19:02:29 smithi149 bash[64262]: Trying to pull quay.io/prometheus/node-exporter:v1.5.0... 2024-09-10T19:02:30.342 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:30 smithi149 ceph-mon[33230]: Deploying daemon node-exporter.b on smithi149 2024-09-10T19:02:30.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:30 smithi149 ceph-mon[33230]: pgmap v36: 132 pgs: 4 peering, 128 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:02:30.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:30 smithi149 ceph-mon[33230]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-09-10T19:02:30.343 INFO:journalctl@ceph.node-exporter.b.smithi149.stdout:Sep 10 19:02:29 smithi149 bash[64262]: Getting image source signatures 2024-09-10T19:02:30.343 INFO:journalctl@ceph.node-exporter.b.smithi149.stdout:Sep 10 19:02:29 smithi149 bash[64262]: Copying blob sha256:22b70bddd3acadc892fca4c2af4260629bfda5dfd11ebc106a93ce24e752b5ed 2024-09-10T19:02:30.343 INFO:journalctl@ceph.node-exporter.b.smithi149.stdout:Sep 10 19:02:29 smithi149 bash[64262]: Copying blob sha256:c0e87333d3805a7b7e1c25783dd25f27da0fe70a8e262eee7ed190c8dc843fa7 2024-09-10T19:02:30.344 INFO:journalctl@ceph.node-exporter.b.smithi149.stdout:Sep 10 19:02:29 smithi149 bash[64262]: Copying blob sha256:5c12815fee558b157dee7f7509dedbaba0a8379098858a65ec869e1f1526ea0c 2024-09-10T19:02:30.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:30 smithi047 ceph-mon[29019]: Deploying daemon node-exporter.b on smithi149 2024-09-10T19:02:30.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:30 smithi047 ceph-mon[29019]: pgmap v36: 132 pgs: 4 peering, 128 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:02:30.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:30 smithi047 ceph-mon[29019]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-09-10T19:02:30.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:30 smithi047 ceph-mon[39856]: Deploying daemon node-exporter.b on smithi149 2024-09-10T19:02:30.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:30 smithi047 ceph-mon[39856]: pgmap v36: 132 pgs: 4 peering, 128 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:02:30.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:30 smithi047 ceph-mon[39856]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-09-10T19:02:31.591 INFO:journalctl@ceph.node-exporter.b.smithi149.stdout:Sep 10 19:02:31 smithi149 bash[64262]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-09-10T19:02:31.591 INFO:journalctl@ceph.node-exporter.b.smithi149.stdout:Sep 10 19:02:31 smithi149 bash[64262]: Writing manifest to image destination 2024-09-10T19:02:31.842 INFO:journalctl@ceph.node-exporter.b.smithi149.stdout:Sep 10 19:02:31 smithi149 podman[64262]: 2024-09-10 19:02:31.581525081 +0000 UTC m=+2.211512746 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-09-10T19:02:31.843 INFO:journalctl@ceph.node-exporter.b.smithi149.stdout:Sep 10 19:02:31 smithi149 podman[64262]: 2024-09-10 19:02:31.705254485 +0000 UTC m=+2.335242142 container create 937888306c31cacb2e464ed48db00c47d62b74493c13890a6176a254ed0c0694 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-10T19:02:32.046 INFO:teuthology.orchestra.run.smithi047.stderr:++ ceph auth get-key osd.0 2024-09-10T19:02:32.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:32 smithi149 ceph-mon[33230]: pgmap v37: 132 pgs: 4 peering, 128 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:02:32.344 INFO:journalctl@ceph.node-exporter.b.smithi149.stdout:Sep 10 19:02:31 smithi149 podman[64262]: 2024-09-10 19:02:31.940639935 +0000 UTC m=+2.570627590 container init 937888306c31cacb2e464ed48db00c47d62b74493c13890a6176a254ed0c0694 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-10T19:02:32.344 INFO:journalctl@ceph.node-exporter.b.smithi149.stdout:Sep 10 19:02:31 smithi149 podman[64262]: 2024-09-10 19:02:31.94345055 +0000 UTC m=+2.573438206 container start 937888306c31cacb2e464ed48db00c47d62b74493c13890a6176a254ed0c0694 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-10T19:02:32.344 INFO:journalctl@ceph.node-exporter.b.smithi149.stdout:Sep 10 19:02:31 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-b[64339]: ts=2024-09-10T19:02:31.947Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-09-10T19:02:32.344 INFO:journalctl@ceph.node-exporter.b.smithi149.stdout:Sep 10 19:02:31 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-b[64339]: ts=2024-09-10T19:02:31.947Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-09-10T19:02:32.344 INFO:journalctl@ceph.node-exporter.b.smithi149.stdout:Sep 10 19:02:31 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-b[64339]: ts=2024-09-10T19:02:31.947Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-09-10T19:02:32.344 INFO:journalctl@ceph.node-exporter.b.smithi149.stdout:Sep 10 19:02:31 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-b[64339]: ts=2024-09-10T19:02:31.947Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-09-10T19:02:32.344 INFO:journalctl@ceph.node-exporter.b.smithi149.stdout:Sep 10 19:02:31 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-b[64339]: ts=2024-09-10T19:02:31.947Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-09-10T19:02:32.344 INFO:journalctl@ceph.node-exporter.b.smithi149.stdout:Sep 10 19:02:31 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-b[64339]: ts=2024-09-10T19:02:31.948Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-09-10T19:02:32.344 INFO:journalctl@ceph.node-exporter.b.smithi149.stdout:Sep 10 19:02:31 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-b[64339]: ts=2024-09-10T19:02:31.948Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-09-10T19:02:32.345 INFO:journalctl@ceph.node-exporter.b.smithi149.stdout:Sep 10 19:02:31 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-b[64339]: ts=2024-09-10T19:02:31.948Z caller=node_exporter.go:117 level=info collector=arp 2024-09-10T19:02:32.345 INFO:journalctl@ceph.node-exporter.b.smithi149.stdout:Sep 10 19:02:31 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-b[64339]: ts=2024-09-10T19:02:31.948Z caller=node_exporter.go:117 level=info collector=bcache 2024-09-10T19:02:32.345 INFO:journalctl@ceph.node-exporter.b.smithi149.stdout:Sep 10 19:02:31 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-b[64339]: ts=2024-09-10T19:02:31.948Z caller=node_exporter.go:117 level=info collector=bonding 2024-09-10T19:02:32.345 INFO:journalctl@ceph.node-exporter.b.smithi149.stdout:Sep 10 19:02:31 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-b[64339]: ts=2024-09-10T19:02:31.948Z caller=node_exporter.go:117 level=info collector=btrfs 2024-09-10T19:02:32.345 INFO:journalctl@ceph.node-exporter.b.smithi149.stdout:Sep 10 19:02:31 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-b[64339]: ts=2024-09-10T19:02:31.948Z caller=node_exporter.go:117 level=info collector=conntrack 2024-09-10T19:02:32.345 INFO:journalctl@ceph.node-exporter.b.smithi149.stdout:Sep 10 19:02:31 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-b[64339]: ts=2024-09-10T19:02:31.948Z caller=node_exporter.go:117 level=info collector=cpu 2024-09-10T19:02:32.345 INFO:journalctl@ceph.node-exporter.b.smithi149.stdout:Sep 10 19:02:31 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-b[64339]: ts=2024-09-10T19:02:31.948Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-09-10T19:02:32.345 INFO:journalctl@ceph.node-exporter.b.smithi149.stdout:Sep 10 19:02:31 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-b[64339]: ts=2024-09-10T19:02:31.948Z caller=node_exporter.go:117 level=info collector=diskstats 2024-09-10T19:02:32.345 INFO:journalctl@ceph.node-exporter.b.smithi149.stdout:Sep 10 19:02:31 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-b[64339]: ts=2024-09-10T19:02:31.948Z caller=node_exporter.go:117 level=info collector=dmi 2024-09-10T19:02:32.345 INFO:journalctl@ceph.node-exporter.b.smithi149.stdout:Sep 10 19:02:31 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-b[64339]: ts=2024-09-10T19:02:31.948Z caller=node_exporter.go:117 level=info collector=edac 2024-09-10T19:02:32.345 INFO:journalctl@ceph.node-exporter.b.smithi149.stdout:Sep 10 19:02:31 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-b[64339]: ts=2024-09-10T19:02:31.948Z caller=node_exporter.go:117 level=info collector=entropy 2024-09-10T19:02:32.346 INFO:journalctl@ceph.node-exporter.b.smithi149.stdout:Sep 10 19:02:31 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-b[64339]: ts=2024-09-10T19:02:31.948Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-09-10T19:02:32.346 INFO:journalctl@ceph.node-exporter.b.smithi149.stdout:Sep 10 19:02:31 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-b[64339]: ts=2024-09-10T19:02:31.948Z caller=node_exporter.go:117 level=info collector=filefd 2024-09-10T19:02:32.346 INFO:journalctl@ceph.node-exporter.b.smithi149.stdout:Sep 10 19:02:31 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-b[64339]: ts=2024-09-10T19:02:31.948Z caller=node_exporter.go:117 level=info collector=filesystem 2024-09-10T19:02:32.346 INFO:journalctl@ceph.node-exporter.b.smithi149.stdout:Sep 10 19:02:31 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-b[64339]: ts=2024-09-10T19:02:31.948Z caller=node_exporter.go:117 level=info collector=hwmon 2024-09-10T19:02:32.346 INFO:journalctl@ceph.node-exporter.b.smithi149.stdout:Sep 10 19:02:31 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-b[64339]: ts=2024-09-10T19:02:31.948Z caller=node_exporter.go:117 level=info collector=infiniband 2024-09-10T19:02:32.346 INFO:journalctl@ceph.node-exporter.b.smithi149.stdout:Sep 10 19:02:31 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-b[64339]: ts=2024-09-10T19:02:31.948Z caller=node_exporter.go:117 level=info collector=ipvs 2024-09-10T19:02:32.346 INFO:journalctl@ceph.node-exporter.b.smithi149.stdout:Sep 10 19:02:31 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-b[64339]: ts=2024-09-10T19:02:31.948Z caller=node_exporter.go:117 level=info collector=loadavg 2024-09-10T19:02:32.346 INFO:journalctl@ceph.node-exporter.b.smithi149.stdout:Sep 10 19:02:31 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-b[64339]: ts=2024-09-10T19:02:31.948Z caller=node_exporter.go:117 level=info collector=mdadm 2024-09-10T19:02:32.347 INFO:journalctl@ceph.node-exporter.b.smithi149.stdout:Sep 10 19:02:31 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-b[64339]: ts=2024-09-10T19:02:31.948Z caller=node_exporter.go:117 level=info collector=meminfo 2024-09-10T19:02:32.347 INFO:journalctl@ceph.node-exporter.b.smithi149.stdout:Sep 10 19:02:31 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-b[64339]: ts=2024-09-10T19:02:31.948Z caller=node_exporter.go:117 level=info collector=netclass 2024-09-10T19:02:32.347 INFO:journalctl@ceph.node-exporter.b.smithi149.stdout:Sep 10 19:02:31 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-b[64339]: ts=2024-09-10T19:02:31.948Z caller=node_exporter.go:117 level=info collector=netdev 2024-09-10T19:02:32.347 INFO:journalctl@ceph.node-exporter.b.smithi149.stdout:Sep 10 19:02:31 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-b[64339]: ts=2024-09-10T19:02:31.948Z caller=node_exporter.go:117 level=info collector=netstat 2024-09-10T19:02:32.347 INFO:journalctl@ceph.node-exporter.b.smithi149.stdout:Sep 10 19:02:31 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-b[64339]: ts=2024-09-10T19:02:31.948Z caller=node_exporter.go:117 level=info collector=nfs 2024-09-10T19:02:32.347 INFO:journalctl@ceph.node-exporter.b.smithi149.stdout:Sep 10 19:02:31 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-b[64339]: ts=2024-09-10T19:02:31.948Z caller=node_exporter.go:117 level=info collector=nfsd 2024-09-10T19:02:32.347 INFO:journalctl@ceph.node-exporter.b.smithi149.stdout:Sep 10 19:02:31 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-b[64339]: ts=2024-09-10T19:02:31.948Z caller=node_exporter.go:117 level=info collector=nvme 2024-09-10T19:02:32.347 INFO:journalctl@ceph.node-exporter.b.smithi149.stdout:Sep 10 19:02:31 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-b[64339]: ts=2024-09-10T19:02:31.948Z caller=node_exporter.go:117 level=info collector=os 2024-09-10T19:02:32.347 INFO:journalctl@ceph.node-exporter.b.smithi149.stdout:Sep 10 19:02:31 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-b[64339]: ts=2024-09-10T19:02:31.948Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-09-10T19:02:32.348 INFO:journalctl@ceph.node-exporter.b.smithi149.stdout:Sep 10 19:02:31 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-b[64339]: ts=2024-09-10T19:02:31.948Z caller=node_exporter.go:117 level=info collector=pressure 2024-09-10T19:02:32.348 INFO:journalctl@ceph.node-exporter.b.smithi149.stdout:Sep 10 19:02:31 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-b[64339]: ts=2024-09-10T19:02:31.948Z caller=node_exporter.go:117 level=info collector=rapl 2024-09-10T19:02:32.348 INFO:journalctl@ceph.node-exporter.b.smithi149.stdout:Sep 10 19:02:31 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-b[64339]: ts=2024-09-10T19:02:31.948Z caller=node_exporter.go:117 level=info collector=schedstat 2024-09-10T19:02:32.348 INFO:journalctl@ceph.node-exporter.b.smithi149.stdout:Sep 10 19:02:31 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-b[64339]: ts=2024-09-10T19:02:31.948Z caller=node_exporter.go:117 level=info collector=selinux 2024-09-10T19:02:32.348 INFO:journalctl@ceph.node-exporter.b.smithi149.stdout:Sep 10 19:02:31 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-b[64339]: ts=2024-09-10T19:02:31.948Z caller=node_exporter.go:117 level=info collector=sockstat 2024-09-10T19:02:32.348 INFO:journalctl@ceph.node-exporter.b.smithi149.stdout:Sep 10 19:02:31 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-b[64339]: ts=2024-09-10T19:02:31.948Z caller=node_exporter.go:117 level=info collector=softnet 2024-09-10T19:02:32.348 INFO:journalctl@ceph.node-exporter.b.smithi149.stdout:Sep 10 19:02:31 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-b[64339]: ts=2024-09-10T19:02:31.948Z caller=node_exporter.go:117 level=info collector=stat 2024-09-10T19:02:32.348 INFO:journalctl@ceph.node-exporter.b.smithi149.stdout:Sep 10 19:02:31 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-b[64339]: ts=2024-09-10T19:02:31.948Z caller=node_exporter.go:117 level=info collector=tapestats 2024-09-10T19:02:32.348 INFO:journalctl@ceph.node-exporter.b.smithi149.stdout:Sep 10 19:02:31 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-b[64339]: ts=2024-09-10T19:02:31.948Z caller=node_exporter.go:117 level=info collector=textfile 2024-09-10T19:02:32.348 INFO:journalctl@ceph.node-exporter.b.smithi149.stdout:Sep 10 19:02:31 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-b[64339]: ts=2024-09-10T19:02:31.948Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-09-10T19:02:32.348 INFO:journalctl@ceph.node-exporter.b.smithi149.stdout:Sep 10 19:02:31 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-b[64339]: ts=2024-09-10T19:02:31.948Z caller=node_exporter.go:117 level=info collector=time 2024-09-10T19:02:32.348 INFO:journalctl@ceph.node-exporter.b.smithi149.stdout:Sep 10 19:02:31 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-b[64339]: ts=2024-09-10T19:02:31.948Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-09-10T19:02:32.349 INFO:journalctl@ceph.node-exporter.b.smithi149.stdout:Sep 10 19:02:31 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-b[64339]: ts=2024-09-10T19:02:31.948Z caller=node_exporter.go:117 level=info collector=uname 2024-09-10T19:02:32.349 INFO:journalctl@ceph.node-exporter.b.smithi149.stdout:Sep 10 19:02:31 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-b[64339]: ts=2024-09-10T19:02:31.948Z caller=node_exporter.go:117 level=info collector=vmstat 2024-09-10T19:02:32.349 INFO:journalctl@ceph.node-exporter.b.smithi149.stdout:Sep 10 19:02:31 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-b[64339]: ts=2024-09-10T19:02:31.948Z caller=node_exporter.go:117 level=info collector=xfs 2024-09-10T19:02:32.349 INFO:journalctl@ceph.node-exporter.b.smithi149.stdout:Sep 10 19:02:31 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-b[64339]: ts=2024-09-10T19:02:31.948Z caller=node_exporter.go:117 level=info collector=zfs 2024-09-10T19:02:32.349 INFO:journalctl@ceph.node-exporter.b.smithi149.stdout:Sep 10 19:02:31 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-b[64339]: ts=2024-09-10T19:02:31.949Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-09-10T19:02:32.349 INFO:journalctl@ceph.node-exporter.b.smithi149.stdout:Sep 10 19:02:31 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-b[64339]: ts=2024-09-10T19:02:31.949Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-09-10T19:02:32.349 INFO:journalctl@ceph.node-exporter.b.smithi149.stdout:Sep 10 19:02:32 smithi149 bash[64262]: 937888306c31cacb2e464ed48db00c47d62b74493c13890a6176a254ed0c0694 2024-09-10T19:02:32.349 INFO:journalctl@ceph.node-exporter.b.smithi149.stdout:Sep 10 19:02:32 smithi149 systemd[1]: Started Ceph node-exporter.b for d8563d7e-6fa5-11ef-bcea-c7b262605968. 2024-09-10T19:02:32.384 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:32 smithi047 ceph-mon[29019]: pgmap v37: 132 pgs: 4 peering, 128 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:02:32.384 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:32 smithi047 ceph-mon[39856]: pgmap v37: 132 pgs: 4 peering, 128 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:02:32.385 INFO:teuthology.orchestra.run.smithi047.stderr:+ NK=AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== 2024-09-10T19:02:32.385 INFO:teuthology.orchestra.run.smithi047.stderr:+ '[' AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== == AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== ']' 2024-09-10T19:02:32.385 INFO:teuthology.orchestra.run.smithi047.stderr:+ sleep 5 2024-09-10T19:02:33.070 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:02:32 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: ::ffff:172.21.15.149 - - [10/Sep/2024:19:02:32] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:02:33.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:33 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:33.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:33 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:33.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:33 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:33.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:33 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:33.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:33 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:33.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:33 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:02:33.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:33 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/3895847274' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:02:33.539 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:33 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:33.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:33 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:33.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:33 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:33.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:33 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:33.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:33 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:33.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:33 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:02:33.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:33 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/3895847274' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:02:33.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:33 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:33.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:33 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:33.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:33 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:33.541 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:33 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:33.541 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:33 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:33.541 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:33 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:02:33.541 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:33 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/3895847274' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:02:34.401 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:34 smithi149 ceph-mon[33230]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 58 B/s, 2 objects/s recovering 2024-09-10T19:02:34.401 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:34 smithi149 ceph-mon[33230]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-09-10T19:02:34.402 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:34 smithi149 ceph-mon[33230]: Cluster is now healthy 2024-09-10T19:02:34.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:34 smithi047 ceph-mon[29019]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 58 B/s, 2 objects/s recovering 2024-09-10T19:02:34.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:34 smithi047 ceph-mon[29019]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-09-10T19:02:34.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:34 smithi047 ceph-mon[29019]: Cluster is now healthy 2024-09-10T19:02:34.465 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:34 smithi047 ceph-mon[39856]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 58 B/s, 2 objects/s recovering 2024-09-10T19:02:34.465 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:34 smithi047 ceph-mon[39856]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-09-10T19:02:34.465 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:34 smithi047 ceph-mon[39856]: Cluster is now healthy 2024-09-10T19:02:36.163 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:35 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:36.163 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:35 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:36.163 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:35 smithi149 ceph-mon[33230]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 58 B/s, 2 objects/s recovering 2024-09-10T19:02:36.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:35 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:36.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:35 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:36.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:35 smithi047 ceph-mon[29019]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 58 B/s, 2 objects/s recovering 2024-09-10T19:02:36.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:35 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:36.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:35 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:36.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:35 smithi047 ceph-mon[39856]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 58 B/s, 2 objects/s recovering 2024-09-10T19:02:37.386 INFO:teuthology.orchestra.run.smithi047.stderr:++ ceph auth get-key osd.0 2024-09-10T19:02:37.643 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:37 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:37.643 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:37 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:37.643 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:37 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:02:37.643 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:37 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:02:37.643 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:37 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:37.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:37 smithi149 ceph-mon[33230]: Rotating authentication key for osd.0 2024-09-10T19:02:37.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:37 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.0", "format": "json"}]: dispatch 2024-09-10T19:02:37.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:37 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.0", "format": "json"}]': finished 2024-09-10T19:02:37.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:37 smithi149 ceph-mon[33230]: Reconfiguring daemon osd.0 on smithi047 2024-09-10T19:02:37.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:37 smithi149 ceph-mon[33230]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s; 47 B/s, 1 objects/s recovering 2024-09-10T19:02:37.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:37 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:37.714 INFO:teuthology.orchestra.run.smithi047.stderr:+ NK=AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== 2024-09-10T19:02:37.715 INFO:teuthology.orchestra.run.smithi047.stderr:+ '[' AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== == AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== ']' 2024-09-10T19:02:37.715 INFO:teuthology.orchestra.run.smithi047.stderr:+ sleep 5 2024-09-10T19:02:37.715 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:37 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:37.715 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:37 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:37.715 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:37 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:02:37.715 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:37 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:02:37.715 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:37 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:37.715 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:37 smithi047 ceph-mon[29019]: Rotating authentication key for osd.0 2024-09-10T19:02:37.716 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:37 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.0", "format": "json"}]: dispatch 2024-09-10T19:02:37.716 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:37 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.0", "format": "json"}]': finished 2024-09-10T19:02:37.716 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:37 smithi047 ceph-mon[29019]: Reconfiguring daemon osd.0 on smithi047 2024-09-10T19:02:37.716 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:37 smithi047 ceph-mon[29019]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s; 47 B/s, 1 objects/s recovering 2024-09-10T19:02:37.716 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:37 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:37.716 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:37 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:37.716 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:37 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:37.716 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:37 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:02:37.716 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:37 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:02:37.716 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:37 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:37.716 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:37 smithi047 ceph-mon[39856]: Rotating authentication key for osd.0 2024-09-10T19:02:37.717 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:37 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.0", "format": "json"}]: dispatch 2024-09-10T19:02:37.717 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:37 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.0", "format": "json"}]': finished 2024-09-10T19:02:37.717 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:37 smithi047 ceph-mon[39856]: Reconfiguring daemon osd.0 on smithi047 2024-09-10T19:02:37.717 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:37 smithi047 ceph-mon[39856]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s; 47 B/s, 1 objects/s recovering 2024-09-10T19:02:37.717 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:37 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:38.343 INFO:journalctl@ceph.iscsi.iscsi.a.smithi149.stdout:Sep 10 19:02:38 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-iscsi-iscsi-a[60538]: debug there is no tcmu-runner data available 2024-09-10T19:02:38.789 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:38 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/3955088912' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:02:38.789 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:38 smithi047 ceph-mon[39856]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:02:38.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:38 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:38.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:38 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:38.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:38 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/3955088912' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:02:38.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:38 smithi047 ceph-mon[29019]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:02:38.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:38 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:38.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:38 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:38.843 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:38 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/3955088912' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:02:38.843 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:38 smithi149 ceph-mon[33230]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:02:38.843 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:38 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:38.843 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:38 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:39.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:39 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:39.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:39 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:39.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:39 smithi047 ceph-mon[29019]: Reconfiguring alertmanager.a (dependencies changed)... 2024-09-10T19:02:39.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:39 smithi047 ceph-mon[29019]: Reconfiguring daemon alertmanager.a on smithi047 2024-09-10T19:02:39.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:39 smithi047 ceph-mon[29019]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.9 KiB/s rd, 373 B/s wr, 7 op/s; 42 B/s, 1 objects/s recovering 2024-09-10T19:02:39.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:39 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:39.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:39 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:39.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:39 smithi047 ceph-mon[39856]: Reconfiguring alertmanager.a (dependencies changed)... 2024-09-10T19:02:39.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:39 smithi047 ceph-mon[39856]: Reconfiguring daemon alertmanager.a on smithi047 2024-09-10T19:02:39.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:39 smithi047 ceph-mon[39856]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.9 KiB/s rd, 373 B/s wr, 7 op/s; 42 B/s, 1 objects/s recovering 2024-09-10T19:02:39.843 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:39 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:39.843 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:39 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:39.844 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:39 smithi149 ceph-mon[33230]: Reconfiguring alertmanager.a (dependencies changed)... 2024-09-10T19:02:39.844 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:39 smithi149 ceph-mon[33230]: Reconfiguring daemon alertmanager.a on smithi047 2024-09-10T19:02:39.844 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:39 smithi149 ceph-mon[33230]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.9 KiB/s rd, 373 B/s wr, 7 op/s; 42 B/s, 1 objects/s recovering 2024-09-10T19:02:41.892 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Sep 10 19:02:41 smithi047 systemd[1]: Stopping Ceph alertmanager.a for d8563d7e-6fa5-11ef-bcea-c7b262605968... 2024-09-10T19:02:42.209 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:42 smithi047 ceph-mon[29019]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.6 KiB/s rd, 341 B/s wr, 6 op/s; 38 B/s, 1 objects/s recovering 2024-09-10T19:02:42.209 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:42 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:02:42.209 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:42 smithi047 ceph-mon[39856]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.6 KiB/s rd, 341 B/s wr, 6 op/s; 38 B/s, 1 objects/s recovering 2024-09-10T19:02:42.209 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:42 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:02:42.210 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Sep 10 19:02:41 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-alertmanager-a[72004]: ts=2024-09-10T19:02:41.890Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-09-10T19:02:42.210 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Sep 10 19:02:41 smithi047 podman[73957]: 2024-09-10 19:02:41.931862634 +0000 UTC m=+0.148111220 container died 57a030f6a82d678974abee6a091611f6addc20d9efcd6bee1291b5ea7bee255f (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-10T19:02:42.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:42 smithi149 ceph-mon[33230]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.6 KiB/s rd, 341 B/s wr, 6 op/s; 38 B/s, 1 objects/s recovering 2024-09-10T19:02:42.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:42 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:02:42.539 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Sep 10 19:02:42 smithi047 podman[73957]: 2024-09-10 19:02:42.316565191 +0000 UTC m=+0.532813776 container cleanup 57a030f6a82d678974abee6a091611f6addc20d9efcd6bee1291b5ea7bee255f (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-10T19:02:42.539 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Sep 10 19:02:42 smithi047 bash[73957]: ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-alertmanager-a 2024-09-10T19:02:42.717 INFO:teuthology.orchestra.run.smithi047.stderr:++ ceph auth get-key osd.0 2024-09-10T19:02:43.021 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Sep 10 19:02:42 smithi047 podman[73970]: 2024-09-10 19:02:42.552311104 +0000 UTC m=+0.623440160 container remove 57a030f6a82d678974abee6a091611f6addc20d9efcd6bee1291b5ea7bee255f (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-10T19:02:43.021 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Sep 10 19:02:42 smithi047 podman[73970]: 2024-09-10 19:02:42.636298516 +0000 UTC m=+0.707427572 volume remove 7cd1d79ab56a38550595937a54ec91027b7e5ea2188a6a77f45236773d57d931 2024-09-10T19:02:43.021 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Sep 10 19:02:42 smithi047 systemd[1]: ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968@alertmanager.a.service: Deactivated successfully. 2024-09-10T19:02:43.022 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Sep 10 19:02:42 smithi047 systemd[1]: Stopped Ceph alertmanager.a for d8563d7e-6fa5-11ef-bcea-c7b262605968. 2024-09-10T19:02:43.022 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Sep 10 19:02:42 smithi047 systemd[1]: Starting Ceph alertmanager.a for d8563d7e-6fa5-11ef-bcea-c7b262605968... 2024-09-10T19:02:43.071 INFO:teuthology.orchestra.run.smithi047.stderr:+ NK=AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== 2024-09-10T19:02:43.071 INFO:teuthology.orchestra.run.smithi047.stderr:+ '[' AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== == AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== ']' 2024-09-10T19:02:43.071 INFO:teuthology.orchestra.run.smithi047.stderr:+ sleep 5 2024-09-10T19:02:43.289 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:43 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:43.289 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:43 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/3276616402' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:02:43.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:43 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:43.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:43 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/3276616402' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:02:43.290 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Sep 10 19:02:43 smithi047 podman[74079]: 2024-09-10 19:02:42.923408831 +0000 UTC m=+0.016461982 image pull c8568f914cd25b2062c44e9f79f9c18da6e3b85fe0c47a12a2191c61426c2b19 quay.io/prometheus/alertmanager:v0.25.0 2024-09-10T19:02:43.290 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Sep 10 19:02:43 smithi047 podman[74079]: 2024-09-10 19:02:43.021525509 +0000 UTC m=+0.114578650 volume create 3c70c8373c8b39b09d56ec093a63b3acb2a1de00fc23932fec2bc3e9dc6bc2d8 2024-09-10T19:02:43.290 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Sep 10 19:02:43 smithi047 podman[74079]: 2024-09-10 19:02:43.153927435 +0000 UTC m=+0.246980574 container create 197b90c742dbe35cf70ec0f3bba1dd69430403fc972014b6a9b4b9e6895e2808 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-10T19:02:43.343 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:02:42 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: ::ffff:172.21.15.149 - - [10/Sep/2024:19:02:42] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:02:43.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:43 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:43.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:43 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/3276616402' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:02:43.790 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Sep 10 19:02:43 smithi047 podman[74079]: 2024-09-10 19:02:43.435869193 +0000 UTC m=+0.528922332 container init 197b90c742dbe35cf70ec0f3bba1dd69430403fc972014b6a9b4b9e6895e2808 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-10T19:02:43.790 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Sep 10 19:02:43 smithi047 podman[74079]: 2024-09-10 19:02:43.438799107 +0000 UTC m=+0.531852255 container start 197b90c742dbe35cf70ec0f3bba1dd69430403fc972014b6a9b4b9e6895e2808 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-10T19:02:43.790 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Sep 10 19:02:43 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-alertmanager-a[74096]: ts=2024-09-10T19:02:43.451Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=HEAD, revision=258fab7cdd551f2cf251ed0348f0ad7289aee789)" 2024-09-10T19:02:43.790 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Sep 10 19:02:43 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-alertmanager-a[74096]: ts=2024-09-10T19:02:43.451Z caller=main.go:241 level=info build_context="(go=go1.19.4, user=root@abe866dd5717, date=20221222-14:51:36)" 2024-09-10T19:02:43.790 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Sep 10 19:02:43 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-alertmanager-a[74096]: ts=2024-09-10T19:02:43.452Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.47 port=9094 2024-09-10T19:02:43.790 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Sep 10 19:02:43 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-alertmanager-a[74096]: ts=2024-09-10T19:02:43.453Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-09-10T19:02:43.790 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Sep 10 19:02:43 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-alertmanager-a[74096]: ts=2024-09-10T19:02:43.476Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-10T19:02:43.790 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Sep 10 19:02:43 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-alertmanager-a[74096]: ts=2024-09-10T19:02:43.476Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-10T19:02:43.790 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Sep 10 19:02:43 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-alertmanager-a[74096]: ts=2024-09-10T19:02:43.479Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9093 2024-09-10T19:02:43.790 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Sep 10 19:02:43 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-alertmanager-a[74096]: ts=2024-09-10T19:02:43.479Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-09-10T19:02:43.790 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Sep 10 19:02:43 smithi047 bash[74079]: 197b90c742dbe35cf70ec0f3bba1dd69430403fc972014b6a9b4b9e6895e2808 2024-09-10T19:02:43.790 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Sep 10 19:02:43 smithi047 systemd[1]: Started Ceph alertmanager.a for d8563d7e-6fa5-11ef-bcea-c7b262605968. 2024-09-10T19:02:44.342 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:44 smithi149 ceph-mon[33230]: pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 4.0 KiB/s rd, 341 B/s wr, 7 op/s; 38 B/s, 1 objects/s recovering 2024-09-10T19:02:44.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:44 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:44.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:44 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:44.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:44 smithi047 ceph-mon[29019]: pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 4.0 KiB/s rd, 341 B/s wr, 7 op/s; 38 B/s, 1 objects/s recovering 2024-09-10T19:02:44.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:44 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:44.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:44 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:44.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:44 smithi047 ceph-mon[39856]: pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 4.0 KiB/s rd, 341 B/s wr, 7 op/s; 38 B/s, 1 objects/s recovering 2024-09-10T19:02:44.541 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:44 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:44.541 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:44 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:45.455 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:45 smithi047 ceph-mon[29019]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-10T19:02:45.455 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:45 smithi047 ceph-mon[29019]: Reconfiguring daemon prometheus.a on smithi149 2024-09-10T19:02:45.455 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:45 smithi047 ceph-mon[39856]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-10T19:02:45.455 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:45 smithi047 ceph-mon[39856]: Reconfiguring daemon prometheus.a on smithi149 2024-09-10T19:02:45.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:45 smithi149 ceph-mon[33230]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-10T19:02:45.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:45 smithi149 ceph-mon[33230]: Reconfiguring daemon prometheus.a on smithi149 2024-09-10T19:02:45.789 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Sep 10 19:02:45 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-alertmanager-a[74096]: ts=2024-09-10T19:02:45.454Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.00096154s 2024-09-10T19:02:46.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:46 smithi047 ceph-mon[29019]: pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:02:46.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:46 smithi047 ceph-mon[39856]: pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:02:46.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:46 smithi149 ceph-mon[33230]: pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:02:48.073 INFO:teuthology.orchestra.run.smithi047.stderr:++ ceph auth get-key osd.0 2024-09-10T19:02:48.343 INFO:journalctl@ceph.iscsi.iscsi.a.smithi149.stdout:Sep 10 19:02:48 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-iscsi-iscsi-a[60538]: debug there is no tcmu-runner data available 2024-09-10T19:02:48.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:48 smithi149 ceph-mon[33230]: pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:02:48.401 INFO:teuthology.orchestra.run.smithi047.stderr:+ NK=AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== 2024-09-10T19:02:48.401 INFO:teuthology.orchestra.run.smithi047.stderr:+ '[' AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== == AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== ']' 2024-09-10T19:02:48.401 INFO:teuthology.orchestra.run.smithi047.stderr:+ sleep 5 2024-09-10T19:02:48.401 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:48 smithi047 ceph-mon[29019]: pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:02:48.402 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:48 smithi047 ceph-mon[39856]: pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:02:49.238 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:49 smithi149 ceph-mon[33230]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:02:49.238 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:49 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/446445670' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:02:49.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:49 smithi047 ceph-mon[29019]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:02:49.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:49 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/446445670' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:02:49.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:49 smithi047 ceph-mon[39856]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:02:49.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:49 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/446445670' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:02:50.217 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:50 smithi149 ceph-mon[33230]: pgmap v46: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:02:50.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:50 smithi047 ceph-mon[29019]: pgmap v46: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:02:50.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:50 smithi047 ceph-mon[39856]: pgmap v46: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:02:51.543 INFO:journalctl@ceph.prometheus.a.smithi149.stdout:Sep 10 19:02:51 smithi149 systemd[1]: Stopping Ceph prometheus.a for d8563d7e-6fa5-11ef-bcea-c7b262605968... 2024-09-10T19:02:51.794 INFO:journalctl@ceph.prometheus.a.smithi149.stdout:Sep 10 19:02:51 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-prometheus-a[62170]: ts=2024-09-10T19:02:51.668Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-09-10T19:02:51.794 INFO:journalctl@ceph.prometheus.a.smithi149.stdout:Sep 10 19:02:51 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-prometheus-a[62170]: ts=2024-09-10T19:02:51.668Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-09-10T19:02:51.794 INFO:journalctl@ceph.prometheus.a.smithi149.stdout:Sep 10 19:02:51 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-prometheus-a[62170]: ts=2024-09-10T19:02:51.668Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-09-10T19:02:51.794 INFO:journalctl@ceph.prometheus.a.smithi149.stdout:Sep 10 19:02:51 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-prometheus-a[62170]: ts=2024-09-10T19:02:51.668Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-09-10T19:02:51.794 INFO:journalctl@ceph.prometheus.a.smithi149.stdout:Sep 10 19:02:51 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-prometheus-a[62170]: ts=2024-09-10T19:02:51.668Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-09-10T19:02:51.794 INFO:journalctl@ceph.prometheus.a.smithi149.stdout:Sep 10 19:02:51 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-prometheus-a[62170]: ts=2024-09-10T19:02:51.668Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-09-10T19:02:51.794 INFO:journalctl@ceph.prometheus.a.smithi149.stdout:Sep 10 19:02:51 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-prometheus-a[62170]: ts=2024-09-10T19:02:51.668Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-09-10T19:02:51.795 INFO:journalctl@ceph.prometheus.a.smithi149.stdout:Sep 10 19:02:51 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-prometheus-a[62170]: ts=2024-09-10T19:02:51.668Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-09-10T19:02:51.795 INFO:journalctl@ceph.prometheus.a.smithi149.stdout:Sep 10 19:02:51 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-prometheus-a[62170]: ts=2024-09-10T19:02:51.669Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-09-10T19:02:51.795 INFO:journalctl@ceph.prometheus.a.smithi149.stdout:Sep 10 19:02:51 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-prometheus-a[62170]: ts=2024-09-10T19:02:51.669Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-09-10T19:02:51.795 INFO:journalctl@ceph.prometheus.a.smithi149.stdout:Sep 10 19:02:51 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-prometheus-a[62170]: ts=2024-09-10T19:02:51.669Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-09-10T19:02:51.795 INFO:journalctl@ceph.prometheus.a.smithi149.stdout:Sep 10 19:02:51 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-prometheus-a[62170]: ts=2024-09-10T19:02:51.669Z caller=main.go:1147 level=info msg="See you next time!" 2024-09-10T19:02:51.795 INFO:journalctl@ceph.prometheus.a.smithi149.stdout:Sep 10 19:02:51 smithi149 podman[65235]: 2024-09-10 19:02:51.679062799 +0000 UTC m=+0.115184171 container died 824cd4ba514effffe1b07406c0011e85c883c29b050eb78e3b8691dcc155fd4d (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-10T19:02:52.093 INFO:journalctl@ceph.prometheus.a.smithi149.stdout:Sep 10 19:02:52 smithi149 podman[65235]: 2024-09-10 19:02:52.01000325 +0000 UTC m=+0.446124614 container cleanup 824cd4ba514effffe1b07406c0011e85c883c29b050eb78e3b8691dcc155fd4d (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-10T19:02:52.093 INFO:journalctl@ceph.prometheus.a.smithi149.stdout:Sep 10 19:02:52 smithi149 bash[65235]: ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-prometheus-a 2024-09-10T19:02:52.459 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:52 smithi149 ceph-mon[33230]: pgmap v47: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:02:52.459 INFO:journalctl@ceph.prometheus.a.smithi149.stdout:Sep 10 19:02:52 smithi149 podman[65250]: 2024-09-10 19:02:52.228266041 +0000 UTC m=+0.550647177 container remove 824cd4ba514effffe1b07406c0011e85c883c29b050eb78e3b8691dcc155fd4d (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-10T19:02:52.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:52 smithi047 ceph-mon[29019]: pgmap v47: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:02:52.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:52 smithi047 ceph-mon[39856]: pgmap v47: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:02:52.807 INFO:journalctl@ceph.prometheus.a.smithi149.stdout:Sep 10 19:02:52 smithi149 systemd[1]: ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968@prometheus.a.service: Deactivated successfully. 2024-09-10T19:02:52.807 INFO:journalctl@ceph.prometheus.a.smithi149.stdout:Sep 10 19:02:52 smithi149 systemd[1]: Stopped Ceph prometheus.a for d8563d7e-6fa5-11ef-bcea-c7b262605968. 2024-09-10T19:02:52.807 INFO:journalctl@ceph.prometheus.a.smithi149.stdout:Sep 10 19:02:52 smithi149 systemd[1]: ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968@prometheus.a.service: Consumed 1.083s CPU time. 2024-09-10T19:02:52.808 INFO:journalctl@ceph.prometheus.a.smithi149.stdout:Sep 10 19:02:52 smithi149 systemd[1]: Starting Ceph prometheus.a for d8563d7e-6fa5-11ef-bcea-c7b262605968... 2024-09-10T19:02:52.808 INFO:journalctl@ceph.prometheus.a.smithi149.stdout:Sep 10 19:02:52 smithi149 podman[65349]: 2024-09-10 19:02:52.58863069 +0000 UTC m=+0.020105189 image pull a07b618ecd1dce142bce4c52f0e80982eaf1f14265a2415c2d35978ccaa0a464 quay.io/prometheus/prometheus:v2.43.0 2024-09-10T19:02:52.808 INFO:journalctl@ceph.prometheus.a.smithi149.stdout:Sep 10 19:02:52 smithi149 podman[65349]: 2024-09-10 19:02:52.687552198 +0000 UTC m=+0.119026673 container create d54b0a8cd0949c243715347ede78bc2eefe91d84159da84c9e535101d433df50 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-10T19:02:53.093 INFO:journalctl@ceph.prometheus.a.smithi149.stdout:Sep 10 19:02:52 smithi149 podman[65349]: 2024-09-10 19:02:52.95663601 +0000 UTC m=+0.388110490 container init d54b0a8cd0949c243715347ede78bc2eefe91d84159da84c9e535101d433df50 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-10T19:02:53.093 INFO:journalctl@ceph.prometheus.a.smithi149.stdout:Sep 10 19:02:52 smithi149 podman[65349]: 2024-09-10 19:02:52.959736972 +0000 UTC m=+0.391211448 container start d54b0a8cd0949c243715347ede78bc2eefe91d84159da84c9e535101d433df50 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-10T19:02:53.093 INFO:journalctl@ceph.prometheus.a.smithi149.stdout:Sep 10 19:02:52 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-prometheus-a[65365]: ts=2024-09-10T19:02:52.983Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2024-09-10T19:02:53.094 INFO:journalctl@ceph.prometheus.a.smithi149.stdout:Sep 10 19:02:52 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-prometheus-a[65365]: ts=2024-09-10T19:02:52.983Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2024-09-10T19:02:53.094 INFO:journalctl@ceph.prometheus.a.smithi149.stdout:Sep 10 19:02:52 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-prometheus-a[65365]: ts=2024-09-10T19:02:52.983Z caller=main.go:570 level=info host_details="(Linux 5.14.0-505.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Sep 5 07:54:07 UTC 2024 x86_64 smithi149 (none))" 2024-09-10T19:02:53.094 INFO:journalctl@ceph.prometheus.a.smithi149.stdout:Sep 10 19:02:52 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-prometheus-a[65365]: ts=2024-09-10T19:02:52.983Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-09-10T19:02:53.094 INFO:journalctl@ceph.prometheus.a.smithi149.stdout:Sep 10 19:02:52 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-prometheus-a[65365]: ts=2024-09-10T19:02:52.983Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-09-10T19:02:53.094 INFO:journalctl@ceph.prometheus.a.smithi149.stdout:Sep 10 19:02:52 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-prometheus-a[65365]: ts=2024-09-10T19:02:52.985Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2024-09-10T19:02:53.094 INFO:journalctl@ceph.prometheus.a.smithi149.stdout:Sep 10 19:02:52 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-prometheus-a[65365]: ts=2024-09-10T19:02:52.985Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2024-09-10T19:02:53.094 INFO:journalctl@ceph.prometheus.a.smithi149.stdout:Sep 10 19:02:52 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-prometheus-a[65365]: ts=2024-09-10T19:02:52.987Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2024-09-10T19:02:53.094 INFO:journalctl@ceph.prometheus.a.smithi149.stdout:Sep 10 19:02:52 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-prometheus-a[65365]: ts=2024-09-10T19:02:52.987Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-09-10T19:02:53.094 INFO:journalctl@ceph.prometheus.a.smithi149.stdout:Sep 10 19:02:52 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-prometheus-a[65365]: ts=2024-09-10T19:02:52.988Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-09-10T19:02:53.094 INFO:journalctl@ceph.prometheus.a.smithi149.stdout:Sep 10 19:02:52 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-prometheus-a[65365]: ts=2024-09-10T19:02:52.988Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=1.518µs 2024-09-10T19:02:53.094 INFO:journalctl@ceph.prometheus.a.smithi149.stdout:Sep 10 19:02:52 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-prometheus-a[65365]: ts=2024-09-10T19:02:52.988Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-09-10T19:02:53.095 INFO:journalctl@ceph.prometheus.a.smithi149.stdout:Sep 10 19:02:52 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-prometheus-a[65365]: ts=2024-09-10T19:02:52.989Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=1 2024-09-10T19:02:53.095 INFO:journalctl@ceph.prometheus.a.smithi149.stdout:Sep 10 19:02:52 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-prometheus-a[65365]: ts=2024-09-10T19:02:52.989Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=1 maxSegment=1 2024-09-10T19:02:53.095 INFO:journalctl@ceph.prometheus.a.smithi149.stdout:Sep 10 19:02:52 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-prometheus-a[65365]: ts=2024-09-10T19:02:52.989Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=30.528µs wal_replay_duration=504.553µs wbl_replay_duration=168ns total_replay_duration=558.277µs 2024-09-10T19:02:53.095 INFO:journalctl@ceph.prometheus.a.smithi149.stdout:Sep 10 19:02:52 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-prometheus-a[65365]: ts=2024-09-10T19:02:52.990Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2024-09-10T19:02:53.095 INFO:journalctl@ceph.prometheus.a.smithi149.stdout:Sep 10 19:02:52 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-prometheus-a[65365]: ts=2024-09-10T19:02:52.990Z caller=main.go:1029 level=info msg="TSDB started" 2024-09-10T19:02:53.095 INFO:journalctl@ceph.prometheus.a.smithi149.stdout:Sep 10 19:02:52 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-prometheus-a[65365]: ts=2024-09-10T19:02:52.990Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-09-10T19:02:53.095 INFO:journalctl@ceph.prometheus.a.smithi149.stdout:Sep 10 19:02:53 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-prometheus-a[65365]: ts=2024-09-10T19:02:53.004Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=13.913623ms db_storage=709ns remote_storage=1.169µs web_handler=221ns query_engine=629ns scrape=598.739µs scrape_sd=26.874µs notify=21.064µs notify_sd=9.996µs rules=13.022391ms tracing=5.933µs 2024-09-10T19:02:53.095 INFO:journalctl@ceph.prometheus.a.smithi149.stdout:Sep 10 19:02:53 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-prometheus-a[65365]: ts=2024-09-10T19:02:53.004Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-09-10T19:02:53.095 INFO:journalctl@ceph.prometheus.a.smithi149.stdout:Sep 10 19:02:53 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-prometheus-a[65365]: ts=2024-09-10T19:02:53.004Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-09-10T19:02:53.095 INFO:journalctl@ceph.prometheus.a.smithi149.stdout:Sep 10 19:02:53 smithi149 bash[65349]: d54b0a8cd0949c243715347ede78bc2eefe91d84159da84c9e535101d433df50 2024-09-10T19:02:53.095 INFO:journalctl@ceph.prometheus.a.smithi149.stdout:Sep 10 19:02:53 smithi149 systemd[1]: Started Ceph prometheus.a for d8563d7e-6fa5-11ef-bcea-c7b262605968. 2024-09-10T19:02:53.397 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:02:53 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: [10/Sep/2024:19:02:53] ENGINE Bus STOPPING 2024-09-10T19:02:53.397 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:02:53 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: [10/Sep/2024:19:02:53] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-10T19:02:53.397 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:02:53 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: [10/Sep/2024:19:02:53] ENGINE Bus STOPPED 2024-09-10T19:02:53.398 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:02:53 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: [10/Sep/2024:19:02:53] ENGINE Bus STARTING 2024-09-10T19:02:53.403 INFO:teuthology.orchestra.run.smithi047.stderr:++ ceph auth get-key osd.0 2024-09-10T19:02:53.693 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Sep 10 19:02:53 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-alertmanager-a[74096]: ts=2024-09-10T19:02:53.457Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.003731657s 2024-09-10T19:02:53.693 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:02:53 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: [10/Sep/2024:19:02:53] ENGINE Serving on http://:::9283 2024-09-10T19:02:53.693 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:02:53 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: [10/Sep/2024:19:02:53] ENGINE Bus STARTED 2024-09-10T19:02:53.693 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:02:53 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: [10/Sep/2024:19:02:53] ENGINE Bus STOPPING 2024-09-10T19:02:53.751 INFO:teuthology.orchestra.run.smithi047.stderr:+ NK=AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== 2024-09-10T19:02:53.751 INFO:teuthology.orchestra.run.smithi047.stderr:+ '[' AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== == AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== ']' 2024-09-10T19:02:53.751 INFO:teuthology.orchestra.run.smithi047.stderr:+ sleep 5 2024-09-10T19:02:54.289 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:02:54 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: [10/Sep/2024:19:02:54] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-10T19:02:54.290 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:02:54 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: [10/Sep/2024:19:02:54] ENGINE Bus STOPPED 2024-09-10T19:02:54.290 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:02:54 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: [10/Sep/2024:19:02:54] ENGINE Bus STARTING 2024-09-10T19:02:54.290 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:02:54 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: [10/Sep/2024:19:02:54] ENGINE Serving on http://:::9283 2024-09-10T19:02:54.290 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:02:54 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: [10/Sep/2024:19:02:54] ENGINE Bus STARTED 2024-09-10T19:02:54.290 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:02:54 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: [10/Sep/2024:19:02:54] ENGINE Bus STOPPING 2024-09-10T19:02:54.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:54 smithi047 ceph-mon[39856]: pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:02:54.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:54 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:54.291 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:54 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:54.291 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:54 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-10T19:02:54.291 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:54 smithi047 ceph-mon[39856]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-10T19:02:54.291 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:54 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi047.front.sepia.ceph.com:9093"}]: dispatch 2024-09-10T19:02:54.291 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:54 smithi047 ceph-mon[39856]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi047.front.sepia.ceph.com:9093"}]: dispatch 2024-09-10T19:02:54.291 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:54 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:54.291 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:54 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-10T19:02:54.291 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:54 smithi047 ceph-mon[39856]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-10T19:02:54.291 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:54 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi149.front.sepia.ceph.com:9095"}]: dispatch 2024-09-10T19:02:54.291 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:54 smithi047 ceph-mon[39856]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi149.front.sepia.ceph.com:9095"}]: dispatch 2024-09-10T19:02:54.291 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:54 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:54.291 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:54 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-10T19:02:54.291 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:54 smithi047 ceph-mon[39856]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-10T19:02:54.292 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:54 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi149.front.sepia.ceph.com:3000"}]: dispatch 2024-09-10T19:02:54.292 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:54 smithi047 ceph-mon[39856]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi149.front.sepia.ceph.com:3000"}]: dispatch 2024-09-10T19:02:54.292 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:54 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:54.292 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:54 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:02:54.292 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:54 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/34353352' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:02:54.292 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:54 smithi047 ceph-mon[29019]: pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:02:54.292 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:54 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:54.292 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:54 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:54.292 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:54 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-10T19:02:54.292 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:54 smithi047 ceph-mon[29019]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-10T19:02:54.292 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:54 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi047.front.sepia.ceph.com:9093"}]: dispatch 2024-09-10T19:02:54.293 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:54 smithi047 ceph-mon[29019]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi047.front.sepia.ceph.com:9093"}]: dispatch 2024-09-10T19:02:54.293 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:54 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:54.293 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:54 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-10T19:02:54.293 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:54 smithi047 ceph-mon[29019]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-10T19:02:54.293 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:54 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi149.front.sepia.ceph.com:9095"}]: dispatch 2024-09-10T19:02:54.293 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:54 smithi047 ceph-mon[29019]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi149.front.sepia.ceph.com:9095"}]: dispatch 2024-09-10T19:02:54.293 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:54 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:54.293 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:54 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-10T19:02:54.293 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:54 smithi047 ceph-mon[29019]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-10T19:02:54.293 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:54 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi149.front.sepia.ceph.com:3000"}]: dispatch 2024-09-10T19:02:54.293 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:54 smithi047 ceph-mon[29019]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi149.front.sepia.ceph.com:3000"}]: dispatch 2024-09-10T19:02:54.293 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:54 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:54.293 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:54 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:02:54.293 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:54 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/34353352' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:02:54.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:54 smithi149 ceph-mon[33230]: pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:02:54.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:54 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:54.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:54 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:54.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:54 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-10T19:02:54.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:54 smithi149 ceph-mon[33230]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-10T19:02:54.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:54 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi047.front.sepia.ceph.com:9093"}]: dispatch 2024-09-10T19:02:54.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:54 smithi149 ceph-mon[33230]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi047.front.sepia.ceph.com:9093"}]: dispatch 2024-09-10T19:02:54.344 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:54 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:54.344 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:54 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-10T19:02:54.344 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:54 smithi149 ceph-mon[33230]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-10T19:02:54.344 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:54 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi149.front.sepia.ceph.com:9095"}]: dispatch 2024-09-10T19:02:54.344 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:54 smithi149 ceph-mon[33230]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi149.front.sepia.ceph.com:9095"}]: dispatch 2024-09-10T19:02:54.344 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:54 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:54.344 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:54 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-10T19:02:54.344 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:54 smithi149 ceph-mon[33230]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-10T19:02:54.344 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:54 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi149.front.sepia.ceph.com:3000"}]: dispatch 2024-09-10T19:02:54.344 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:54 smithi149 ceph-mon[33230]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi149.front.sepia.ceph.com:3000"}]: dispatch 2024-09-10T19:02:54.344 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:54 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:54.344 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:54 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:02:54.344 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:54 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/34353352' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:02:54.903 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:02:54 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: [10/Sep/2024:19:02:54] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-10T19:02:54.903 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:02:54 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: [10/Sep/2024:19:02:54] ENGINE Bus STOPPED 2024-09-10T19:02:54.903 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:02:54 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: [10/Sep/2024:19:02:54] ENGINE Bus STARTING 2024-09-10T19:02:54.903 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:02:54 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: [10/Sep/2024:19:02:54] ENGINE Serving on http://:::9283 2024-09-10T19:02:54.903 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:02:54 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: [10/Sep/2024:19:02:54] ENGINE Bus STARTED 2024-09-10T19:02:56.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:56 smithi149 ceph-mon[33230]: pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:02:56.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:56 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:56.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:56 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:56.539 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:56 smithi047 ceph-mon[39856]: pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:02:56.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:56 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:56.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:56 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:56.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:56 smithi047 ceph-mon[29019]: pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:02:56.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:56 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:56.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:56 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:57.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:57 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:02:57.539 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:57 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:02:57.592 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:57 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:02:58.343 INFO:journalctl@ceph.iscsi.iscsi.a.smithi149.stdout:Sep 10 19:02:58 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-iscsi-iscsi-a[60538]: debug there is no tcmu-runner data available 2024-09-10T19:02:58.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:58 smithi149 ceph-mon[33230]: pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:02:58.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:58 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:58.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:58 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:58.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:58 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:02:58.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:58 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:02:58.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:58 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:58.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:58 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:02:58.539 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:58 smithi047 ceph-mon[39856]: pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:02:58.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:58 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:58.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:58 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:58.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:58 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:02:58.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:58 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:02:58.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:58 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:58.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:58 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:02:58.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:58 smithi047 ceph-mon[29019]: pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:02:58.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:58 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:58.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:58 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:58.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:58 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:02:58.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:58 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:02:58.541 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:58 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:02:58.541 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:58 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:02:58.753 INFO:teuthology.orchestra.run.smithi047.stderr:++ ceph auth get-key osd.0 2024-09-10T19:02:59.073 INFO:teuthology.orchestra.run.smithi047.stderr:+ NK=AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== 2024-09-10T19:02:59.073 INFO:teuthology.orchestra.run.smithi047.stderr:+ '[' AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== == AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== ']' 2024-09-10T19:02:59.073 INFO:teuthology.orchestra.run.smithi047.stderr:+ sleep 5 2024-09-10T19:02:59.289 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:59 smithi047 ceph-mon[29019]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:02:59.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:02:59 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/1834810975' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:02:59.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:59 smithi047 ceph-mon[39856]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:02:59.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:02:59 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/1834810975' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:02:59.497 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:59 smithi149 ceph-mon[33230]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:02:59.497 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:02:59 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/1834810975' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:03:00.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:03:00 smithi149 ceph-mon[33230]: pgmap v51: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:03:00.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:03:00 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:03:00.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:03:00 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:03:00.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:03:00 smithi047 ceph-mon[29019]: pgmap v51: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:03:00.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:03:00 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:03:00.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:03:00 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:03:00.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:03:00 smithi047 ceph-mon[39856]: pgmap v51: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:03:00.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:03:00 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:03:00.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:03:00 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:03:02.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:03:02 smithi047 ceph-mon[29019]: pgmap v52: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:03:02.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:03:02 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:03:02.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:03:02 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:03:02.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:03:02 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:03:02.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:03:02 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:03:02.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:03:02 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:03:02.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:03:02 smithi047 ceph-mon[39856]: pgmap v52: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:03:02.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:03:02 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:03:02.541 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:03:02 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:03:02.541 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:03:02 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:03:02.541 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:03:02 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:03:02.541 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:03:02 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:03:02.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:03:02 smithi149 ceph-mon[33230]: pgmap v52: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:03:02.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:03:02 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:03:02.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:03:02 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:03:02.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:03:02 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:03:02.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:03:02 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:03:02.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:03:02 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:03:03.343 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:03:02 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: ::ffff:172.21.15.149 - - [10/Sep/2024:19:03:02] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:03:04.075 INFO:teuthology.orchestra.run.smithi047.stderr:++ ceph auth get-key osd.0 2024-09-10T19:03:04.404 INFO:teuthology.orchestra.run.smithi047.stderr:+ NK=AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== 2024-09-10T19:03:04.404 INFO:teuthology.orchestra.run.smithi047.stderr:+ '[' AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== == AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== ']' 2024-09-10T19:03:04.405 INFO:teuthology.orchestra.run.smithi047.stderr:+ sleep 5 2024-09-10T19:03:04.405 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:03:04 smithi047 ceph-mon[29019]: pgmap v53: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:03:04.405 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:03:04 smithi047 ceph-mon[39856]: pgmap v53: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:03:04.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:03:04 smithi149 ceph-mon[33230]: pgmap v53: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:03:05.539 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:03:05 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/11034298' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:03:05.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:03:05 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/11034298' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:03:05.592 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:03:05 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/11034298' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:03:06.039 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:03:05 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: ::ffff:172.21.15.149 - - [10/Sep/2024:19:03:05] "GET /metrics HTTP/1.1" 200 219482 "" "Prometheus/2.43.0" 2024-09-10T19:03:06.326 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:03:06 smithi047 ceph-mon[29019]: pgmap v54: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:03:06.327 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:03:06 smithi047 ceph-mon[39856]: pgmap v54: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:03:06.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:03:06 smithi149 ceph-mon[33230]: pgmap v54: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:03:08.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:03:08 smithi149 ceph-mon[33230]: pgmap v55: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:03:08.343 INFO:journalctl@ceph.iscsi.iscsi.a.smithi149.stdout:Sep 10 19:03:08 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-iscsi-iscsi-a[60538]: debug there is no tcmu-runner data available 2024-09-10T19:03:08.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:03:08 smithi047 ceph-mon[29019]: pgmap v55: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:03:08.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:03:08 smithi047 ceph-mon[39856]: pgmap v55: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:03:09.407 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:03:09 smithi047 ceph-mon[39856]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:03:09.407 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:03:09 smithi047 ceph-mon[29019]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:03:09.407 INFO:teuthology.orchestra.run.smithi047.stderr:++ ceph auth get-key osd.0 2024-09-10T19:03:09.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:03:09 smithi149 ceph-mon[33230]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:03:09.735 INFO:teuthology.orchestra.run.smithi047.stderr:+ NK=AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== 2024-09-10T19:03:09.736 INFO:teuthology.orchestra.run.smithi047.stderr:+ '[' AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== == AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== ']' 2024-09-10T19:03:09.736 INFO:teuthology.orchestra.run.smithi047.stderr:+ sleep 5 2024-09-10T19:03:10.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:03:10 smithi047 ceph-mon[29019]: pgmap v56: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:03:10.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:03:10 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/4174348725' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:03:10.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:03:10 smithi047 ceph-mon[39856]: pgmap v56: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:03:10.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:03:10 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/4174348725' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:03:10.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:03:10 smithi149 ceph-mon[33230]: pgmap v56: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:03:10.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:03:10 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/4174348725' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:03:12.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:03:12 smithi047 ceph-mon[29019]: pgmap v57: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:03:12.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:03:12 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:03:12.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:03:12 smithi047 ceph-mon[39856]: pgmap v57: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:03:12.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:03:12 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:03:12.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:03:12 smithi149 ceph-mon[33230]: pgmap v57: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:03:12.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:03:12 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:03:13.343 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:03:12 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: ::ffff:172.21.15.149 - - [10/Sep/2024:19:03:12] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:03:14.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:03:14 smithi047 ceph-mon[29019]: pgmap v58: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:03:14.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:03:14 smithi047 ceph-mon[39856]: pgmap v58: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:03:14.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:03:14 smithi149 ceph-mon[33230]: pgmap v58: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:03:14.737 INFO:teuthology.orchestra.run.smithi047.stderr:++ ceph auth get-key osd.0 2024-09-10T19:03:15.064 INFO:teuthology.orchestra.run.smithi047.stderr:+ NK=AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== 2024-09-10T19:03:15.065 INFO:teuthology.orchestra.run.smithi047.stderr:+ '[' AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== == AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== ']' 2024-09-10T19:03:15.065 INFO:teuthology.orchestra.run.smithi047.stderr:+ sleep 5 2024-09-10T19:03:15.539 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:03:15 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/3002175408' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:03:15.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:03:15 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/3002175408' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:03:15.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:03:15 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/3002175408' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:03:16.039 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:03:15 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: ::ffff:172.21.15.149 - - [10/Sep/2024:19:03:15] "GET /metrics HTTP/1.1" 200 219459 "" "Prometheus/2.43.0" 2024-09-10T19:03:16.539 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:03:16 smithi047 ceph-mon[39856]: pgmap v59: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:03:16.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:03:16 smithi047 ceph-mon[29019]: pgmap v59: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:03:16.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:03:16 smithi149 ceph-mon[33230]: pgmap v59: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:03:18.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:03:18 smithi047 ceph-mon[29019]: pgmap v60: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:03:18.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:03:18 smithi047 ceph-mon[39856]: pgmap v60: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:03:18.593 INFO:journalctl@ceph.iscsi.iscsi.a.smithi149.stdout:Sep 10 19:03:18 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-iscsi-iscsi-a[60538]: debug there is no tcmu-runner data available 2024-09-10T19:03:18.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:03:18 smithi149 ceph-mon[33230]: pgmap v60: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:03:19.494 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:03:19 smithi149 ceph-mon[33230]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:03:19.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:03:19 smithi047 ceph-mon[29019]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:03:19.539 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:03:19 smithi047 ceph-mon[39856]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:03:20.067 INFO:teuthology.orchestra.run.smithi047.stderr:++ ceph auth get-key osd.0 2024-09-10T19:03:20.401 INFO:teuthology.orchestra.run.smithi047.stderr:+ NK=AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== 2024-09-10T19:03:20.401 INFO:teuthology.orchestra.run.smithi047.stderr:+ '[' AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== == AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== ']' 2024-09-10T19:03:20.401 INFO:teuthology.orchestra.run.smithi047.stderr:+ sleep 5 2024-09-10T19:03:20.401 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:03:20 smithi047 ceph-mon[29019]: pgmap v61: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:03:20.402 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:03:20 smithi047 ceph-mon[39856]: pgmap v61: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:03:20.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:03:20 smithi149 ceph-mon[33230]: pgmap v61: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:03:21.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:03:21 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/3248179248' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:03:21.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:03:21 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/3248179248' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:03:21.592 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:03:21 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/3248179248' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:03:22.539 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:03:22 smithi047 ceph-mon[39856]: pgmap v62: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:03:22.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:03:22 smithi047 ceph-mon[29019]: pgmap v62: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:03:22.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:03:22 smithi149 ceph-mon[33230]: pgmap v62: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:03:23.343 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:03:22 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: ::ffff:172.21.15.149 - - [10/Sep/2024:19:03:22] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:03:24.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:03:24 smithi047 ceph-mon[29019]: pgmap v63: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:03:24.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:03:24 smithi047 ceph-mon[39856]: pgmap v63: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:03:24.592 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:03:24 smithi149 ceph-mon[33230]: pgmap v63: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:03:25.402 INFO:teuthology.orchestra.run.smithi047.stderr:++ ceph auth get-key osd.0 2024-09-10T19:03:25.732 INFO:teuthology.orchestra.run.smithi047.stderr:+ NK=AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== 2024-09-10T19:03:25.732 INFO:teuthology.orchestra.run.smithi047.stderr:+ '[' AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== == AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== ']' 2024-09-10T19:03:25.732 INFO:teuthology.orchestra.run.smithi047.stderr:+ sleep 5 2024-09-10T19:03:26.039 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:03:25 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: ::ffff:172.21.15.149 - - [10/Sep/2024:19:03:25] "GET /metrics HTTP/1.1" 200 219459 "" "Prometheus/2.43.0" 2024-09-10T19:03:26.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:03:26 smithi047 ceph-mon[29019]: pgmap v64: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:03:26.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:03:26 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/597655498' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:03:26.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:03:26 smithi047 ceph-mon[39856]: pgmap v64: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:03:26.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:03:26 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/597655498' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:03:26.592 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:03:26 smithi149 ceph-mon[33230]: pgmap v64: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:03:26.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:03:26 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/597655498' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:03:27.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:03:27 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:03:27.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:03:27 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:03:27.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:03:27 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:03:28.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:03:28 smithi047 ceph-mon[29019]: pgmap v65: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:03:28.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:03:28 smithi047 ceph-mon[39856]: pgmap v65: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:03:28.593 INFO:journalctl@ceph.iscsi.iscsi.a.smithi149.stdout:Sep 10 19:03:28 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-iscsi-iscsi-a[60538]: debug there is no tcmu-runner data available 2024-09-10T19:03:28.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:03:28 smithi149 ceph-mon[33230]: pgmap v65: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:03:29.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:03:29 smithi047 ceph-mon[29019]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:03:29.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:03:29 smithi047 ceph-mon[39856]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:03:29.592 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:03:29 smithi149 ceph-mon[33230]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:03:30.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:03:30 smithi047 ceph-mon[29019]: pgmap v66: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:03:30.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:03:30 smithi047 ceph-mon[39856]: pgmap v66: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:03:30.592 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:03:30 smithi149 ceph-mon[33230]: pgmap v66: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:03:30.734 INFO:teuthology.orchestra.run.smithi047.stderr:++ ceph auth get-key osd.0 2024-09-10T19:03:31.060 INFO:teuthology.orchestra.run.smithi047.stderr:+ NK=AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== 2024-09-10T19:03:31.060 INFO:teuthology.orchestra.run.smithi047.stderr:+ '[' AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== == AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== ']' 2024-09-10T19:03:31.060 INFO:teuthology.orchestra.run.smithi047.stderr:+ sleep 5 2024-09-10T19:03:31.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:03:31 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/3765779712' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:03:31.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:03:31 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/3765779712' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:03:31.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:03:31 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/3765779712' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:03:32.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:03:32 smithi047 ceph-mon[29019]: pgmap v67: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:03:32.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:03:32 smithi047 ceph-mon[39856]: pgmap v67: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:03:32.592 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:03:32 smithi149 ceph-mon[33230]: pgmap v67: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:03:33.343 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:03:32 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: ::ffff:172.21.15.149 - - [10/Sep/2024:19:03:32] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:03:34.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:03:34 smithi047 ceph-mon[29019]: pgmap v68: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:03:34.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:03:34 smithi047 ceph-mon[39856]: pgmap v68: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:03:34.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:03:34 smithi149 ceph-mon[33230]: pgmap v68: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:03:36.039 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:03:35 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: ::ffff:172.21.15.149 - - [10/Sep/2024:19:03:35] "GET /metrics HTTP/1.1" 200 219459 "" "Prometheus/2.43.0" 2024-09-10T19:03:36.061 INFO:teuthology.orchestra.run.smithi047.stderr:++ ceph auth get-key osd.0 2024-09-10T19:03:36.390 INFO:teuthology.orchestra.run.smithi047.stderr:+ NK=AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== 2024-09-10T19:03:36.390 INFO:teuthology.orchestra.run.smithi047.stderr:+ '[' AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== == AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== ']' 2024-09-10T19:03:36.390 INFO:teuthology.orchestra.run.smithi047.stderr:+ sleep 5 2024-09-10T19:03:36.391 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:03:36 smithi047 ceph-mon[29019]: pgmap v69: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:03:36.391 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:03:36 smithi047 ceph-mon[39856]: pgmap v69: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:03:36.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:03:36 smithi149 ceph-mon[33230]: pgmap v69: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:03:37.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:03:37 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/1165620580' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:03:37.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:03:37 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/1165620580' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:03:37.592 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:03:37 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/1165620580' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:03:38.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:03:38 smithi047 ceph-mon[29019]: pgmap v70: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:03:38.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:03:38 smithi047 ceph-mon[39856]: pgmap v70: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:03:38.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:03:38 smithi149 ceph-mon[33230]: pgmap v70: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:03:38.593 INFO:journalctl@ceph.iscsi.iscsi.a.smithi149.stdout:Sep 10 19:03:38 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-iscsi-iscsi-a[60538]: debug there is no tcmu-runner data available 2024-09-10T19:03:39.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:03:39 smithi047 ceph-mon[29019]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:03:39.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:03:39 smithi047 ceph-mon[39856]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:03:39.592 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:03:39 smithi149 ceph-mon[33230]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:03:40.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:03:40 smithi047 ceph-mon[29019]: pgmap v71: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:03:40.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:03:40 smithi047 ceph-mon[39856]: pgmap v71: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:03:40.592 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:03:40 smithi149 ceph-mon[33230]: pgmap v71: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:03:41.392 INFO:teuthology.orchestra.run.smithi047.stderr:++ ceph auth get-key osd.0 2024-09-10T19:03:41.720 INFO:teuthology.orchestra.run.smithi047.stderr:+ NK=AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== 2024-09-10T19:03:41.721 INFO:teuthology.orchestra.run.smithi047.stderr:+ '[' AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== == AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== ']' 2024-09-10T19:03:41.721 INFO:teuthology.orchestra.run.smithi047.stderr:+ sleep 5 2024-09-10T19:03:42.539 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:03:42 smithi047 ceph-mon[39856]: pgmap v72: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:03:42.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:03:42 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/551248477' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:03:42.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:03:42 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:03:42.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:03:42 smithi047 ceph-mon[29019]: pgmap v72: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:03:42.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:03:42 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/551248477' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:03:42.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:03:42 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:03:42.592 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:03:42 smithi149 ceph-mon[33230]: pgmap v72: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:03:42.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:03:42 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/551248477' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:03:42.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:03:42 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:03:43.343 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:03:42 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: ::ffff:172.21.15.149 - - [10/Sep/2024:19:03:42] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:03:44.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:03:44 smithi047 ceph-mon[29019]: pgmap v73: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:03:44.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:03:44 smithi047 ceph-mon[39856]: pgmap v73: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:03:44.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:03:44 smithi149 ceph-mon[33230]: pgmap v73: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:03:46.039 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:03:45 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: ::ffff:172.21.15.149 - - [10/Sep/2024:19:03:45] "GET /metrics HTTP/1.1" 200 219470 "" "Prometheus/2.43.0" 2024-09-10T19:03:46.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:03:46 smithi047 ceph-mon[29019]: pgmap v74: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:03:46.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:03:46 smithi047 ceph-mon[39856]: pgmap v74: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:03:46.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:03:46 smithi149 ceph-mon[33230]: pgmap v74: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:03:46.722 INFO:teuthology.orchestra.run.smithi047.stderr:++ ceph auth get-key osd.0 2024-09-10T19:03:47.051 INFO:teuthology.orchestra.run.smithi047.stderr:+ NK=AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== 2024-09-10T19:03:47.051 INFO:teuthology.orchestra.run.smithi047.stderr:+ '[' AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== == AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== ']' 2024-09-10T19:03:47.051 INFO:teuthology.orchestra.run.smithi047.stderr:+ sleep 5 2024-09-10T19:03:47.539 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:03:47 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/1287616456' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:03:47.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:03:47 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/1287616456' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:03:47.592 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:03:47 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/1287616456' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:03:48.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:03:48 smithi047 ceph-mon[39856]: pgmap v75: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:03:48.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:03:48 smithi047 ceph-mon[29019]: pgmap v75: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:03:48.593 INFO:journalctl@ceph.iscsi.iscsi.a.smithi149.stdout:Sep 10 19:03:48 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-iscsi-iscsi-a[60538]: debug there is no tcmu-runner data available 2024-09-10T19:03:48.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:03:48 smithi149 ceph-mon[33230]: pgmap v75: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:03:49.494 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:03:49 smithi149 ceph-mon[33230]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:03:49.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:03:49 smithi047 ceph-mon[29019]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:03:49.539 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:03:49 smithi047 ceph-mon[39856]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:03:50.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:03:50 smithi047 ceph-mon[29019]: pgmap v76: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:03:50.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:03:50 smithi047 ceph-mon[39856]: pgmap v76: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:03:50.592 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:03:50 smithi149 ceph-mon[33230]: pgmap v76: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:03:52.053 INFO:teuthology.orchestra.run.smithi047.stderr:++ ceph auth get-key osd.0 2024-09-10T19:03:52.385 INFO:teuthology.orchestra.run.smithi047.stderr:+ NK=AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== 2024-09-10T19:03:52.385 INFO:teuthology.orchestra.run.smithi047.stderr:+ '[' AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== == AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== ']' 2024-09-10T19:03:52.385 INFO:teuthology.orchestra.run.smithi047.stderr:+ sleep 5 2024-09-10T19:03:52.386 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:03:52 smithi047 ceph-mon[29019]: pgmap v77: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:03:52.386 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:03:52 smithi047 ceph-mon[39856]: pgmap v77: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:03:52.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:03:52 smithi149 ceph-mon[33230]: pgmap v77: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:03:53.227 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:03:52 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: ::ffff:172.21.15.149 - - [10/Sep/2024:19:03:52] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:03:53.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:03:53 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/253810463' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:03:53.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:03:53 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/253810463' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:03:53.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:03:53 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/253810463' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:03:54.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:03:54 smithi047 ceph-mon[29019]: pgmap v78: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:03:54.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:03:54 smithi047 ceph-mon[39856]: pgmap v78: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:03:54.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:03:54 smithi149 ceph-mon[33230]: pgmap v78: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:03:56.039 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:03:55 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: ::ffff:172.21.15.149 - - [10/Sep/2024:19:03:55] "GET /metrics HTTP/1.1" 200 219470 "" "Prometheus/2.43.0" 2024-09-10T19:03:56.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:03:56 smithi047 ceph-mon[29019]: pgmap v79: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:03:56.539 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:03:56 smithi047 ceph-mon[39856]: pgmap v79: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:03:56.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:03:56 smithi149 ceph-mon[33230]: pgmap v79: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:03:57.384 INFO:teuthology.orchestra.run.smithi047.stderr:++ ceph auth get-key osd.0 2024-09-10T19:03:57.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:03:57 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:03:57.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:03:57 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:03:57.592 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:03:57 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:03:57.710 INFO:teuthology.orchestra.run.smithi047.stderr:+ NK=AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== 2024-09-10T19:03:57.710 INFO:teuthology.orchestra.run.smithi047.stderr:+ '[' AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== == AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== ']' 2024-09-10T19:03:57.710 INFO:teuthology.orchestra.run.smithi047.stderr:+ sleep 5 2024-09-10T19:03:58.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:03:58 smithi047 ceph-mon[29019]: pgmap v80: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:03:58.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:03:58 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/3441069957' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:03:58.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:03:58 smithi047 ceph-mon[39856]: pgmap v80: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:03:58.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:03:58 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/3441069957' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:03:58.592 INFO:journalctl@ceph.iscsi.iscsi.a.smithi149.stdout:Sep 10 19:03:58 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-iscsi-iscsi-a[60538]: debug there is no tcmu-runner data available 2024-09-10T19:03:58.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:03:58 smithi149 ceph-mon[33230]: pgmap v80: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:03:58.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:03:58 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/3441069957' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:03:59.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:03:59 smithi047 ceph-mon[29019]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:03:59.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:03:59 smithi047 ceph-mon[39856]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:03:59.592 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:03:59 smithi149 ceph-mon[33230]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:04:00.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:04:00 smithi047 ceph-mon[29019]: pgmap v81: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:04:00.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:04:00 smithi047 ceph-mon[39856]: pgmap v81: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:04:00.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:04:00 smithi149 ceph-mon[33230]: pgmap v81: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:04:01.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:04:01 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:04:01.539 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:04:01 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:04:01.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:04:01 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:04:02.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:04:02 smithi047 ceph-mon[29019]: pgmap v82: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:04:02.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:04:02 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:04:02.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:04:02 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:04:02.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:04:02 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:04:02.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:04:02 smithi047 ceph-mon[39856]: pgmap v82: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:04:02.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:04:02 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:04:02.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:04:02 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:04:02.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:04:02 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:04:02.592 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:04:02 smithi149 ceph-mon[33230]: pgmap v82: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:04:02.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:04:02 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:04:02.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:04:02 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:04:02.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:04:02 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:04:02.712 INFO:teuthology.orchestra.run.smithi047.stderr:++ ceph auth get-key osd.0 2024-09-10T19:04:03.039 INFO:teuthology.orchestra.run.smithi047.stderr:+ NK=AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== 2024-09-10T19:04:03.039 INFO:teuthology.orchestra.run.smithi047.stderr:+ '[' AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== == AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== ']' 2024-09-10T19:04:03.039 INFO:teuthology.orchestra.run.smithi047.stderr:+ sleep 5 2024-09-10T19:04:03.256 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:04:02 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: ::ffff:172.21.15.149 - - [10/Sep/2024:19:04:02] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:04:03.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:04:03 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/3526624343' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:04:03.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:04:03 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/3526624343' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:04:03.592 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:04:03 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/3526624343' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:04:04.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:04:04 smithi047 ceph-mon[29019]: pgmap v83: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:04:04.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:04:04 smithi047 ceph-mon[39856]: pgmap v83: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:04:04.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:04:04 smithi149 ceph-mon[33230]: pgmap v83: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:04:06.039 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:04:05 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: ::ffff:172.21.15.149 - - [10/Sep/2024:19:04:05] "GET /metrics HTTP/1.1" 200 219474 "" "Prometheus/2.43.0" 2024-09-10T19:04:06.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:04:06 smithi047 ceph-mon[29019]: pgmap v84: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:04:06.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:04:06 smithi047 ceph-mon[39856]: pgmap v84: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:04:06.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:04:06 smithi149 ceph-mon[33230]: pgmap v84: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:04:08.041 INFO:teuthology.orchestra.run.smithi047.stderr:++ ceph auth get-key osd.0 2024-09-10T19:04:08.329 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:04:08 smithi047 ceph-mon[29019]: pgmap v85: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:04:08.329 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:04:08 smithi047 ceph-mon[39856]: pgmap v85: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:04:08.358 INFO:teuthology.orchestra.run.smithi047.stderr:+ NK=AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== 2024-09-10T19:04:08.359 INFO:teuthology.orchestra.run.smithi047.stderr:+ '[' AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== == AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== ']' 2024-09-10T19:04:08.359 INFO:teuthology.orchestra.run.smithi047.stderr:+ sleep 5 2024-09-10T19:04:08.593 INFO:journalctl@ceph.iscsi.iscsi.a.smithi149.stdout:Sep 10 19:04:08 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-iscsi-iscsi-a[60538]: debug there is no tcmu-runner data available 2024-09-10T19:04:08.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:04:08 smithi149 ceph-mon[33230]: pgmap v85: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:04:09.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:04:09 smithi047 ceph-mon[29019]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:04:09.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:04:09 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/3333489475' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:04:09.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:04:09 smithi047 ceph-mon[39856]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:04:09.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:04:09 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/3333489475' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:04:09.592 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:04:09 smithi149 ceph-mon[33230]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:04:09.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:04:09 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/3333489475' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:04:10.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:04:10 smithi047 ceph-mon[29019]: pgmap v86: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:04:10.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:04:10 smithi047 ceph-mon[39856]: pgmap v86: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:04:10.592 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:04:10 smithi149 ceph-mon[33230]: pgmap v86: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:04:12.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:04:12 smithi047 ceph-mon[29019]: pgmap v87: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:04:12.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:04:12 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:04:12.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:04:12 smithi047 ceph-mon[39856]: pgmap v87: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:04:12.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:04:12 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:04:12.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:04:12 smithi149 ceph-mon[33230]: pgmap v87: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:04:12.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:04:12 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:04:13.343 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:04:12 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: ::ffff:172.21.15.149 - - [10/Sep/2024:19:04:12] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:04:13.360 INFO:teuthology.orchestra.run.smithi047.stderr:++ ceph auth get-key osd.0 2024-09-10T19:04:13.689 INFO:teuthology.orchestra.run.smithi047.stderr:+ NK=AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== 2024-09-10T19:04:13.689 INFO:teuthology.orchestra.run.smithi047.stderr:+ '[' AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== == AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== ']' 2024-09-10T19:04:13.690 INFO:teuthology.orchestra.run.smithi047.stderr:+ sleep 5 2024-09-10T19:04:14.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:04:14 smithi047 ceph-mon[29019]: pgmap v88: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:04:14.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:04:14 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/4258971702' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:04:14.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:04:14 smithi047 ceph-mon[39856]: pgmap v88: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:04:14.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:04:14 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/4258971702' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:04:14.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:04:14 smithi149 ceph-mon[33230]: pgmap v88: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:04:14.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:04:14 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/4258971702' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:04:16.039 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:04:15 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: ::ffff:172.21.15.149 - - [10/Sep/2024:19:04:15] "GET /metrics HTTP/1.1" 200 219473 "" "Prometheus/2.43.0" 2024-09-10T19:04:16.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:04:16 smithi047 ceph-mon[29019]: pgmap v89: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:04:16.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:04:16 smithi047 ceph-mon[39856]: pgmap v89: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:04:16.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:04:16 smithi149 ceph-mon[33230]: pgmap v89: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:04:18.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:04:18 smithi047 ceph-mon[29019]: pgmap v90: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:04:18.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:04:18 smithi047 ceph-mon[39856]: pgmap v90: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:04:18.593 INFO:journalctl@ceph.iscsi.iscsi.a.smithi149.stdout:Sep 10 19:04:18 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-iscsi-iscsi-a[60538]: debug there is no tcmu-runner data available 2024-09-10T19:04:18.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:04:18 smithi149 ceph-mon[33230]: pgmap v90: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:04:18.691 INFO:teuthology.orchestra.run.smithi047.stderr:++ ceph auth get-key osd.0 2024-09-10T19:04:19.018 INFO:teuthology.orchestra.run.smithi047.stderr:+ NK=AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== 2024-09-10T19:04:19.018 INFO:teuthology.orchestra.run.smithi047.stderr:+ '[' AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== == AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== ']' 2024-09-10T19:04:19.018 INFO:teuthology.orchestra.run.smithi047.stderr:+ sleep 5 2024-09-10T19:04:19.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:04:19 smithi047 ceph-mon[29019]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:04:19.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:04:19 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/1910641792' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:04:19.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:04:19 smithi047 ceph-mon[39856]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:04:19.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:04:19 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/1910641792' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:04:19.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:04:19 smithi149 ceph-mon[33230]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:04:19.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:04:19 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/1910641792' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:04:20.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:04:20 smithi047 ceph-mon[29019]: pgmap v91: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:04:20.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:04:20 smithi047 ceph-mon[39856]: pgmap v91: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:04:20.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:04:20 smithi149 ceph-mon[33230]: pgmap v91: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:04:22.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:04:22 smithi047 ceph-mon[29019]: pgmap v92: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:04:22.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:04:22 smithi047 ceph-mon[39856]: pgmap v92: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:04:22.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:04:22 smithi149 ceph-mon[33230]: pgmap v92: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:04:23.343 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:04:22 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: ::ffff:172.21.15.149 - - [10/Sep/2024:19:04:22] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:04:24.020 INFO:teuthology.orchestra.run.smithi047.stderr:++ ceph auth get-key osd.0 2024-09-10T19:04:24.351 INFO:teuthology.orchestra.run.smithi047.stderr:+ NK=AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== 2024-09-10T19:04:24.351 INFO:teuthology.orchestra.run.smithi047.stderr:+ '[' AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== == AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== ']' 2024-09-10T19:04:24.351 INFO:teuthology.orchestra.run.smithi047.stderr:+ sleep 5 2024-09-10T19:04:24.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:04:24 smithi149 ceph-mon[33230]: pgmap v93: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:04:24.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:04:24 smithi047 ceph-mon[29019]: pgmap v93: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:04:24.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:04:24 smithi047 ceph-mon[39856]: pgmap v93: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:04:25.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:04:25 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/3107936932' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:04:25.707 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:04:25 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/3107936932' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:04:25.707 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:04:25 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/3107936932' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:04:26.039 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:04:25 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: ::ffff:172.21.15.149 - - [10/Sep/2024:19:04:25] "GET /metrics HTTP/1.1" 200 219473 "" "Prometheus/2.43.0" 2024-09-10T19:04:26.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:04:26 smithi149 ceph-mon[33230]: pgmap v94: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:04:26.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:04:26 smithi047 ceph-mon[29019]: pgmap v94: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:04:26.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:04:26 smithi047 ceph-mon[39856]: pgmap v94: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:04:27.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:04:27 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:04:27.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:04:27 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:04:27.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:04:27 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:04:28.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:04:28 smithi149 ceph-mon[33230]: pgmap v95: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:04:28.593 INFO:journalctl@ceph.iscsi.iscsi.a.smithi149.stdout:Sep 10 19:04:28 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-iscsi-iscsi-a[60538]: debug there is no tcmu-runner data available 2024-09-10T19:04:28.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:04:28 smithi047 ceph-mon[29019]: pgmap v95: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:04:28.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:04:28 smithi047 ceph-mon[39856]: pgmap v95: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:04:29.353 INFO:teuthology.orchestra.run.smithi047.stderr:++ ceph auth get-key osd.0 2024-09-10T19:04:29.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:04:29 smithi149 ceph-mon[33230]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:04:29.679 INFO:teuthology.orchestra.run.smithi047.stderr:+ NK=AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== 2024-09-10T19:04:29.679 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:04:29 smithi047 ceph-mon[29019]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:04:29.679 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:04:29 smithi047 ceph-mon[39856]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:04:29.680 INFO:teuthology.orchestra.run.smithi047.stderr:+ '[' AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== == AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== ']' 2024-09-10T19:04:29.680 INFO:teuthology.orchestra.run.smithi047.stderr:+ sleep 5 2024-09-10T19:04:30.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:04:30 smithi149 ceph-mon[33230]: pgmap v96: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:04:30.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:04:30 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/416679382' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:04:30.789 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:04:30 smithi047 ceph-mon[39856]: pgmap v96: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:04:30.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:04:30 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/416679382' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:04:30.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:04:30 smithi047 ceph-mon[29019]: pgmap v96: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:04:30.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:04:30 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/416679382' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:04:32.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:04:32 smithi149 ceph-mon[33230]: pgmap v97: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:04:32.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:04:32 smithi047 ceph-mon[29019]: pgmap v97: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:04:32.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:04:32 smithi047 ceph-mon[39856]: pgmap v97: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:04:33.343 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:04:32 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: ::ffff:172.21.15.149 - - [10/Sep/2024:19:04:32] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:04:34.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:04:34 smithi149 ceph-mon[33230]: pgmap v98: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:04:34.681 INFO:teuthology.orchestra.run.smithi047.stderr:++ ceph auth get-key osd.0 2024-09-10T19:04:34.681 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:04:34 smithi047 ceph-mon[29019]: pgmap v98: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:04:34.681 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:04:34 smithi047 ceph-mon[39856]: pgmap v98: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:04:35.009 INFO:teuthology.orchestra.run.smithi047.stderr:+ NK=AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== 2024-09-10T19:04:35.009 INFO:teuthology.orchestra.run.smithi047.stderr:+ '[' AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== == AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== ']' 2024-09-10T19:04:35.009 INFO:teuthology.orchestra.run.smithi047.stderr:+ sleep 5 2024-09-10T19:04:35.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:04:35 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/4269665012' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:04:35.707 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:04:35 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/4269665012' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:04:35.707 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:04:35 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/4269665012' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:04:36.040 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:04:35 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: ::ffff:172.21.15.149 - - [10/Sep/2024:19:04:35] "GET /metrics HTTP/1.1" 200 219474 "" "Prometheus/2.43.0" 2024-09-10T19:04:36.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:04:36 smithi149 ceph-mon[33230]: pgmap v99: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:04:36.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:04:36 smithi047 ceph-mon[29019]: pgmap v99: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:04:36.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:04:36 smithi047 ceph-mon[39856]: pgmap v99: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:04:38.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:04:38 smithi149 ceph-mon[33230]: pgmap v100: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:04:38.593 INFO:journalctl@ceph.iscsi.iscsi.a.smithi149.stdout:Sep 10 19:04:38 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-iscsi-iscsi-a[60538]: debug there is no tcmu-runner data available 2024-09-10T19:04:38.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:04:38 smithi047 ceph-mon[29019]: pgmap v100: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:04:38.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:04:38 smithi047 ceph-mon[39856]: pgmap v100: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:04:39.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:04:39 smithi149 ceph-mon[33230]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:04:39.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:04:39 smithi047 ceph-mon[29019]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:04:39.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:04:39 smithi047 ceph-mon[39856]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:04:40.011 INFO:teuthology.orchestra.run.smithi047.stderr:++ ceph auth get-key osd.0 2024-09-10T19:04:40.344 INFO:teuthology.orchestra.run.smithi047.stderr:+ NK=AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== 2024-09-10T19:04:40.345 INFO:teuthology.orchestra.run.smithi047.stderr:+ '[' AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== == AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== ']' 2024-09-10T19:04:40.345 INFO:teuthology.orchestra.run.smithi047.stderr:+ sleep 5 2024-09-10T19:04:40.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:04:40 smithi149 ceph-mon[33230]: pgmap v101: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:04:40.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:04:40 smithi047 ceph-mon[29019]: pgmap v101: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:04:40.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:04:40 smithi047 ceph-mon[39856]: pgmap v101: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:04:41.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:04:41 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/3549373625' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:04:41.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:04:41 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/3549373625' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:04:41.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:04:41 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/3549373625' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:04:42.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:04:42 smithi149 ceph-mon[33230]: pgmap v102: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:04:42.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:04:42 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:04:42.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:04:42 smithi047 ceph-mon[29019]: pgmap v102: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:04:42.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:04:42 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:04:42.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:04:42 smithi047 ceph-mon[39856]: pgmap v102: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:04:42.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:04:42 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:04:43.341 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:04:42 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: ::ffff:172.21.15.149 - - [10/Sep/2024:19:04:42] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:04:43.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:04:43 smithi149 ceph-mon[33230]: pgmap v103: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:04:43.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:04:43 smithi047 ceph-mon[29019]: pgmap v103: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:04:43.789 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:04:43 smithi047 ceph-mon[39856]: pgmap v103: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:04:45.346 INFO:teuthology.orchestra.run.smithi047.stderr:++ ceph auth get-key osd.0 2024-09-10T19:04:45.681 INFO:teuthology.orchestra.run.smithi047.stderr:+ NK=AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== 2024-09-10T19:04:45.681 INFO:teuthology.orchestra.run.smithi047.stderr:+ '[' AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== == AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== ']' 2024-09-10T19:04:45.681 INFO:teuthology.orchestra.run.smithi047.stderr:+ sleep 5 2024-09-10T19:04:46.039 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:04:45 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: ::ffff:172.21.15.149 - - [10/Sep/2024:19:04:45] "GET /metrics HTTP/1.1" 200 219443 "" "Prometheus/2.43.0" 2024-09-10T19:04:46.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:04:46 smithi047 ceph-mon[39856]: pgmap v104: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:04:46.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:04:46 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/1067335431' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:04:46.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:04:46 smithi047 ceph-mon[29019]: pgmap v104: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:04:46.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:04:46 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/1067335431' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:04:46.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:04:46 smithi149 ceph-mon[33230]: pgmap v104: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:04:46.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:04:46 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/1067335431' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:04:48.539 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:04:48 smithi047 ceph-mon[39856]: pgmap v105: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:04:48.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:04:48 smithi047 ceph-mon[29019]: pgmap v105: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:04:48.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:04:48 smithi149 ceph-mon[33230]: pgmap v105: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:04:48.593 INFO:journalctl@ceph.iscsi.iscsi.a.smithi149.stdout:Sep 10 19:04:48 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-iscsi-iscsi-a[60538]: debug there is no tcmu-runner data available 2024-09-10T19:04:49.494 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:04:49 smithi149 ceph-mon[33230]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:04:49.539 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:04:49 smithi047 ceph-mon[39856]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:04:49.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:04:49 smithi047 ceph-mon[29019]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:04:50.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:04:50 smithi047 ceph-mon[39856]: pgmap v106: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:04:50.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:04:50 smithi047 ceph-mon[29019]: pgmap v106: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:04:50.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:04:50 smithi149 ceph-mon[33230]: pgmap v106: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:04:50.683 INFO:teuthology.orchestra.run.smithi047.stderr:++ ceph auth get-key osd.0 2024-09-10T19:04:51.011 INFO:teuthology.orchestra.run.smithi047.stderr:+ NK=AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== 2024-09-10T19:04:51.012 INFO:teuthology.orchestra.run.smithi047.stderr:+ '[' AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== == AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== ']' 2024-09-10T19:04:51.012 INFO:teuthology.orchestra.run.smithi047.stderr:+ sleep 5 2024-09-10T19:04:51.289 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:04:51 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/958634752' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:04:51.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:04:51 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/958634752' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:04:51.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:04:51 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/958634752' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:04:52.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:04:52 smithi047 ceph-mon[29019]: pgmap v107: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:04:52.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:04:52 smithi047 ceph-mon[39856]: pgmap v107: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:04:52.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:04:52 smithi149 ceph-mon[33230]: pgmap v107: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:04:53.343 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:04:52 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: ::ffff:172.21.15.149 - - [10/Sep/2024:19:04:52] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:04:54.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:04:54 smithi047 ceph-mon[29019]: pgmap v108: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:04:54.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:04:54 smithi047 ceph-mon[39856]: pgmap v108: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:04:54.592 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:04:54 smithi149 ceph-mon[33230]: pgmap v108: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:04:56.013 INFO:teuthology.orchestra.run.smithi047.stderr:++ ceph auth get-key osd.0 2024-09-10T19:04:56.014 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:04:55 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: ::ffff:172.21.15.149 - - [10/Sep/2024:19:04:55] "GET /metrics HTTP/1.1" 200 219443 "" "Prometheus/2.43.0" 2024-09-10T19:04:56.289 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:04:56 smithi047 ceph-mon[29019]: pgmap v109: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:04:56.289 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:04:56 smithi047 ceph-mon[39856]: pgmap v109: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:04:56.340 INFO:teuthology.orchestra.run.smithi047.stderr:+ NK=AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== 2024-09-10T19:04:56.340 INFO:teuthology.orchestra.run.smithi047.stderr:+ '[' AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== == AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== ']' 2024-09-10T19:04:56.340 INFO:teuthology.orchestra.run.smithi047.stderr:+ sleep 5 2024-09-10T19:04:56.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:04:56 smithi149 ceph-mon[33230]: pgmap v109: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:04:57.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:04:57 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/2883485865' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:04:57.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:04:57 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:04:57.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:04:57 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/2883485865' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:04:57.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:04:57 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:04:57.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:04:57 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/2883485865' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:04:57.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:04:57 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:04:58.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:04:58 smithi047 ceph-mon[29019]: pgmap v110: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T19:04:58.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:04:58 smithi047 ceph-mon[39856]: pgmap v110: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T19:04:58.592 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:04:58 smithi149 ceph-mon[33230]: pgmap v110: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T19:04:58.593 INFO:journalctl@ceph.iscsi.iscsi.a.smithi149.stdout:Sep 10 19:04:58 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-iscsi-iscsi-a[60538]: debug there is no tcmu-runner data available 2024-09-10T19:04:59.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:04:59 smithi047 ceph-mon[29019]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:04:59.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:04:59 smithi047 ceph-mon[39856]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:04:59.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:04:59 smithi149 ceph-mon[33230]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:05:00.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:05:00 smithi047 ceph-mon[29019]: pgmap v111: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:05:00.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:05:00 smithi047 ceph-mon[39856]: pgmap v111: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:05:00.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:05:00 smithi149 ceph-mon[33230]: pgmap v111: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:05:01.342 INFO:teuthology.orchestra.run.smithi047.stderr:++ ceph auth get-key osd.0 2024-09-10T19:05:01.673 INFO:teuthology.orchestra.run.smithi047.stderr:+ NK=AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== 2024-09-10T19:05:01.673 INFO:teuthology.orchestra.run.smithi047.stderr:+ '[' AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== == AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== ']' 2024-09-10T19:05:01.673 INFO:teuthology.orchestra.run.smithi047.stderr:+ sleep 5 2024-09-10T19:05:02.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:05:02 smithi047 ceph-mon[39856]: pgmap v112: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T19:05:02.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:05:02 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/1171609832' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:05:02.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:05:02 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:05:02.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:05:02 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:05:02.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:05:02 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:05:02.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:05:02 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:05:02.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:05:02 smithi047 ceph-mon[29019]: pgmap v112: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T19:05:02.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:05:02 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/1171609832' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:05:02.541 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:05:02 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:05:02.541 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:05:02 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:05:02.541 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:05:02 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:05:02.541 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:05:02 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:05:02.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:05:02 smithi149 ceph-mon[33230]: pgmap v112: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T19:05:02.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:05:02 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/1171609832' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:05:02.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:05:02 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:05:02.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:05:02 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:05:02.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:05:02 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:05:02.594 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:05:02 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:05:03.343 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:05:02 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: ::ffff:172.21.15.149 - - [10/Sep/2024:19:05:02] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:05:04.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:05:04 smithi047 ceph-mon[29019]: pgmap v113: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:05:04.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:05:04 smithi047 ceph-mon[39856]: pgmap v113: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:05:04.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:05:04 smithi149 ceph-mon[33230]: pgmap v113: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:05:06.039 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:05:05 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: ::ffff:172.21.15.149 - - [10/Sep/2024:19:05:05] "GET /metrics HTTP/1.1" 200 219443 "" "Prometheus/2.43.0" 2024-09-10T19:05:06.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:05:06 smithi047 ceph-mon[29019]: pgmap v114: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T19:05:06.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:05:06 smithi047 ceph-mon[39856]: pgmap v114: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T19:05:06.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:05:06 smithi149 ceph-mon[33230]: pgmap v114: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T19:05:06.675 INFO:teuthology.orchestra.run.smithi047.stderr:++ ceph auth get-key osd.0 2024-09-10T19:05:07.007 INFO:teuthology.orchestra.run.smithi047.stderr:+ NK=AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== 2024-09-10T19:05:07.008 INFO:teuthology.orchestra.run.smithi047.stderr:+ '[' AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== == AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== ']' 2024-09-10T19:05:07.008 INFO:teuthology.orchestra.run.smithi047.stderr:+ sleep 5 2024-09-10T19:05:07.289 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:05:07 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/542852688' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:05:07.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:05:07 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/542852688' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:05:07.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:05:07 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/542852688' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:05:08.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:05:08 smithi047 ceph-mon[29019]: pgmap v115: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T19:05:08.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:05:08 smithi047 ceph-mon[39856]: pgmap v115: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T19:05:08.593 INFO:journalctl@ceph.iscsi.iscsi.a.smithi149.stdout:Sep 10 19:05:08 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-iscsi-iscsi-a[60538]: debug there is no tcmu-runner data available 2024-09-10T19:05:08.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:05:08 smithi149 ceph-mon[33230]: pgmap v115: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T19:05:09.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:05:09 smithi047 ceph-mon[29019]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:05:09.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:05:09 smithi047 ceph-mon[39856]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:05:09.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:05:09 smithi149 ceph-mon[33230]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:05:10.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:05:10 smithi047 ceph-mon[29019]: pgmap v116: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:05:10.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:05:10 smithi047 ceph-mon[39856]: pgmap v116: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:05:10.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:05:10 smithi149 ceph-mon[33230]: pgmap v116: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:05:12.009 INFO:teuthology.orchestra.run.smithi047.stderr:++ ceph auth get-key osd.0 2024-09-10T19:05:12.289 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:05:12 smithi047 ceph-mon[39856]: pgmap v117: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:05:12.289 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:05:12 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:05:12.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:05:12 smithi047 ceph-mon[29019]: pgmap v117: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:05:12.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:05:12 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:05:12.339 INFO:teuthology.orchestra.run.smithi047.stderr:+ NK=AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== 2024-09-10T19:05:12.339 INFO:teuthology.orchestra.run.smithi047.stderr:+ '[' AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== == AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== ']' 2024-09-10T19:05:12.339 INFO:teuthology.orchestra.run.smithi047.stderr:+ sleep 5 2024-09-10T19:05:12.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:05:12 smithi149 ceph-mon[33230]: pgmap v117: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:05:12.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:05:12 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:05:13.179 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:05:12 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: ::ffff:172.21.15.149 - - [10/Sep/2024:19:05:12] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:05:13.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:05:13 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/166020333' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:05:13.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:05:13 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/166020333' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:05:13.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:05:13 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/166020333' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:05:14.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:05:14 smithi047 ceph-mon[29019]: pgmap v118: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:05:14.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:05:14 smithi047 ceph-mon[39856]: pgmap v118: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:05:14.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:05:14 smithi149 ceph-mon[33230]: pgmap v118: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:05:16.039 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:05:15 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: ::ffff:172.21.15.149 - - [10/Sep/2024:19:05:15] "GET /metrics HTTP/1.1" 200 219449 "" "Prometheus/2.43.0" 2024-09-10T19:05:16.539 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:05:16 smithi047 ceph-mon[39856]: pgmap v119: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:05:16.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:05:16 smithi047 ceph-mon[29019]: pgmap v119: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:05:16.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:05:16 smithi149 ceph-mon[33230]: pgmap v119: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:05:17.341 INFO:teuthology.orchestra.run.smithi047.stderr:++ ceph auth get-key osd.0 2024-09-10T19:05:17.671 INFO:teuthology.orchestra.run.smithi047.stderr:+ NK=AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== 2024-09-10T19:05:17.671 INFO:teuthology.orchestra.run.smithi047.stderr:+ '[' AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== == AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== ']' 2024-09-10T19:05:17.672 INFO:teuthology.orchestra.run.smithi047.stderr:+ sleep 5 2024-09-10T19:05:18.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:05:18 smithi047 ceph-mon[29019]: pgmap v120: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:05:18.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:05:18 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/4048734799' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:05:18.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:05:18 smithi047 ceph-mon[39856]: pgmap v120: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:05:18.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:05:18 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/4048734799' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:05:18.593 INFO:journalctl@ceph.iscsi.iscsi.a.smithi149.stdout:Sep 10 19:05:18 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-iscsi-iscsi-a[60538]: debug there is no tcmu-runner data available 2024-09-10T19:05:18.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:05:18 smithi149 ceph-mon[33230]: pgmap v120: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:05:18.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:05:18 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/4048734799' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:05:19.494 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:05:19 smithi149 ceph-mon[33230]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:05:19.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:05:19 smithi047 ceph-mon[29019]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:05:19.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:05:19 smithi047 ceph-mon[39856]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:05:20.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:05:20 smithi047 ceph-mon[29019]: pgmap v121: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:05:20.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:05:20 smithi047 ceph-mon[39856]: pgmap v121: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:05:20.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:05:20 smithi149 ceph-mon[33230]: pgmap v121: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:05:22.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:05:22 smithi047 ceph-mon[29019]: pgmap v122: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:05:22.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:05:22 smithi047 ceph-mon[39856]: pgmap v122: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:05:22.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:05:22 smithi149 ceph-mon[33230]: pgmap v122: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:05:22.673 INFO:teuthology.orchestra.run.smithi047.stderr:++ ceph auth get-key osd.0 2024-09-10T19:05:23.002 INFO:teuthology.orchestra.run.smithi047.stderr:+ NK=AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== 2024-09-10T19:05:23.002 INFO:teuthology.orchestra.run.smithi047.stderr:+ '[' AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== == AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== ']' 2024-09-10T19:05:23.002 INFO:teuthology.orchestra.run.smithi047.stderr:+ sleep 5 2024-09-10T19:05:23.202 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:05:22 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: ::ffff:172.21.15.149 - - [10/Sep/2024:19:05:22] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:05:23.289 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:05:23 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/2335011777' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:05:23.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:05:23 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/2335011777' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:05:23.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:05:23 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/2335011777' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:05:24.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:05:24 smithi047 ceph-mon[29019]: pgmap v123: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:05:24.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:05:24 smithi047 ceph-mon[39856]: pgmap v123: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:05:24.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:05:24 smithi149 ceph-mon[33230]: pgmap v123: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:05:26.039 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:05:25 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: ::ffff:172.21.15.149 - - [10/Sep/2024:19:05:25] "GET /metrics HTTP/1.1" 200 219449 "" "Prometheus/2.43.0" 2024-09-10T19:05:26.539 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:05:26 smithi047 ceph-mon[39856]: pgmap v124: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:05:26.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:05:26 smithi047 ceph-mon[29019]: pgmap v124: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:05:26.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:05:26 smithi149 ceph-mon[33230]: pgmap v124: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:05:27.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:05:27 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:05:27.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:05:27 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:05:27.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:05:27 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:05:28.004 INFO:teuthology.orchestra.run.smithi047.stderr:++ ceph auth get-key osd.0 2024-09-10T19:05:28.289 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:05:28 smithi047 ceph-mon[29019]: pgmap v125: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:05:28.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:05:28 smithi047 ceph-mon[39856]: pgmap v125: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:05:28.335 INFO:teuthology.orchestra.run.smithi047.stderr:+ NK=AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== 2024-09-10T19:05:28.335 INFO:teuthology.orchestra.run.smithi047.stderr:+ '[' AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== == AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== ']' 2024-09-10T19:05:28.335 INFO:teuthology.orchestra.run.smithi047.stderr:+ sleep 5 2024-09-10T19:05:28.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:05:28 smithi149 ceph-mon[33230]: pgmap v125: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:05:28.593 INFO:journalctl@ceph.iscsi.iscsi.a.smithi149.stdout:Sep 10 19:05:28 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-iscsi-iscsi-a[60538]: debug there is no tcmu-runner data available 2024-09-10T19:05:29.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:05:29 smithi047 ceph-mon[29019]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:05:29.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:05:29 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/1717278801' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:05:29.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:05:29 smithi047 ceph-mon[39856]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:05:29.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:05:29 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/1717278801' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:05:29.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:05:29 smithi149 ceph-mon[33230]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:05:29.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:05:29 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/1717278801' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:05:30.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:05:30 smithi047 ceph-mon[29019]: pgmap v126: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:05:30.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:05:30 smithi047 ceph-mon[39856]: pgmap v126: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:05:30.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:05:30 smithi149 ceph-mon[33230]: pgmap v126: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:05:32.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:05:32 smithi047 ceph-mon[29019]: pgmap v127: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:05:32.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:05:32 smithi047 ceph-mon[39856]: pgmap v127: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:05:32.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:05:32 smithi149 ceph-mon[33230]: pgmap v127: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:05:33.337 INFO:teuthology.orchestra.run.smithi047.stderr:++ ceph auth get-key osd.0 2024-09-10T19:05:33.343 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:05:32 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: ::ffff:172.21.15.149 - - [10/Sep/2024:19:05:32] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:05:33.666 INFO:teuthology.orchestra.run.smithi047.stderr:+ NK=AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== 2024-09-10T19:05:33.666 INFO:teuthology.orchestra.run.smithi047.stderr:+ '[' AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== == AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== ']' 2024-09-10T19:05:33.666 INFO:teuthology.orchestra.run.smithi047.stderr:+ sleep 5 2024-09-10T19:05:34.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:05:34 smithi047 ceph-mon[29019]: pgmap v128: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:05:34.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:05:34 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/3790315439' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:05:34.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:05:34 smithi047 ceph-mon[39856]: pgmap v128: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:05:34.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:05:34 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/3790315439' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:05:34.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:05:34 smithi149 ceph-mon[33230]: pgmap v128: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:05:34.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:05:34 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/3790315439' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:05:36.039 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:05:35 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: ::ffff:172.21.15.149 - - [10/Sep/2024:19:05:35] "GET /metrics HTTP/1.1" 200 219452 "" "Prometheus/2.43.0" 2024-09-10T19:05:36.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:05:36 smithi047 ceph-mon[29019]: pgmap v129: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:05:36.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:05:36 smithi047 ceph-mon[39856]: pgmap v129: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:05:36.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:05:36 smithi149 ceph-mon[33230]: pgmap v129: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:05:38.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:05:38 smithi047 ceph-mon[29019]: pgmap v130: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:05:38.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:05:38 smithi047 ceph-mon[39856]: pgmap v130: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:05:38.593 INFO:journalctl@ceph.iscsi.iscsi.a.smithi149.stdout:Sep 10 19:05:38 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-iscsi-iscsi-a[60538]: debug there is no tcmu-runner data available 2024-09-10T19:05:38.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:05:38 smithi149 ceph-mon[33230]: pgmap v130: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:05:38.668 INFO:teuthology.orchestra.run.smithi047.stderr:++ ceph auth get-key osd.0 2024-09-10T19:05:38.997 INFO:teuthology.orchestra.run.smithi047.stderr:+ NK=AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== 2024-09-10T19:05:38.997 INFO:teuthology.orchestra.run.smithi047.stderr:+ '[' AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== == AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== ']' 2024-09-10T19:05:38.998 INFO:teuthology.orchestra.run.smithi047.stderr:+ sleep 5 2024-09-10T19:05:39.289 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:05:39 smithi047 ceph-mon[39856]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:05:39.289 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:05:39 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/2598672664' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:05:39.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:05:39 smithi047 ceph-mon[29019]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:05:39.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:05:39 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/2598672664' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:05:39.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:05:39 smithi149 ceph-mon[33230]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:05:39.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:05:39 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/2598672664' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:05:40.539 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:05:40 smithi047 ceph-mon[39856]: pgmap v131: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:05:40.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:05:40 smithi047 ceph-mon[29019]: pgmap v131: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:05:40.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:05:40 smithi149 ceph-mon[33230]: pgmap v131: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:05:42.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:05:42 smithi047 ceph-mon[39856]: pgmap v132: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:05:42.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:05:42 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:05:42.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:05:42 smithi047 ceph-mon[29019]: pgmap v132: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:05:42.541 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:05:42 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:05:42.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:05:42 smithi149 ceph-mon[33230]: pgmap v132: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:05:42.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:05:42 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:05:43.343 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:05:42 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: ::ffff:172.21.15.149 - - [10/Sep/2024:19:05:42] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:05:43.999 INFO:teuthology.orchestra.run.smithi047.stderr:++ ceph auth get-key osd.0 2024-09-10T19:05:44.289 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:05:44 smithi047 ceph-mon[29019]: pgmap v133: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:05:44.289 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:05:44 smithi047 ceph-mon[39856]: pgmap v133: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:05:44.333 INFO:teuthology.orchestra.run.smithi047.stderr:+ NK=AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== 2024-09-10T19:05:44.333 INFO:teuthology.orchestra.run.smithi047.stderr:+ '[' AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== == AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== ']' 2024-09-10T19:05:44.333 INFO:teuthology.orchestra.run.smithi047.stderr:+ sleep 5 2024-09-10T19:05:44.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:05:44 smithi149 ceph-mon[33230]: pgmap v133: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:05:45.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:05:45 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/43253325' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:05:45.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:05:45 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/43253325' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:05:45.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:05:45 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/43253325' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:05:46.039 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:05:45 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: ::ffff:172.21.15.149 - - [10/Sep/2024:19:05:45] "GET /metrics HTTP/1.1" 200 219458 "" "Prometheus/2.43.0" 2024-09-10T19:05:46.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:05:46 smithi047 ceph-mon[29019]: pgmap v134: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:05:46.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:05:46 smithi047 ceph-mon[39856]: pgmap v134: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:05:46.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:05:46 smithi149 ceph-mon[33230]: pgmap v134: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:05:48.539 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:05:48 smithi047 ceph-mon[39856]: pgmap v135: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:05:48.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:05:48 smithi047 ceph-mon[29019]: pgmap v135: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:05:48.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:05:48 smithi149 ceph-mon[33230]: pgmap v135: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:05:48.593 INFO:journalctl@ceph.iscsi.iscsi.a.smithi149.stdout:Sep 10 19:05:48 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-iscsi-iscsi-a[60538]: debug there is no tcmu-runner data available 2024-09-10T19:05:49.335 INFO:teuthology.orchestra.run.smithi047.stderr:++ ceph auth get-key osd.0 2024-09-10T19:05:49.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:05:49 smithi047 ceph-mon[29019]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:05:49.539 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:05:49 smithi047 ceph-mon[39856]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:05:49.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:05:49 smithi149 ceph-mon[33230]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:05:49.665 INFO:teuthology.orchestra.run.smithi047.stderr:+ NK=AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== 2024-09-10T19:05:49.665 INFO:teuthology.orchestra.run.smithi047.stderr:+ '[' AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== == AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== ']' 2024-09-10T19:05:49.665 INFO:teuthology.orchestra.run.smithi047.stderr:+ sleep 5 2024-09-10T19:05:50.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:05:50 smithi047 ceph-mon[29019]: pgmap v136: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:05:50.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:05:50 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/3858986680' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:05:50.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:05:50 smithi047 ceph-mon[39856]: pgmap v136: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:05:50.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:05:50 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/3858986680' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:05:50.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:05:50 smithi149 ceph-mon[33230]: pgmap v136: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:05:50.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:05:50 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/3858986680' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:05:52.539 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:05:52 smithi047 ceph-mon[39856]: pgmap v137: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:05:52.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:05:52 smithi047 ceph-mon[29019]: pgmap v137: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:05:52.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:05:52 smithi149 ceph-mon[33230]: pgmap v137: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:05:53.343 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:05:52 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: ::ffff:172.21.15.149 - - [10/Sep/2024:19:05:52] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:05:54.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:05:54 smithi047 ceph-mon[29019]: pgmap v138: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:05:54.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:05:54 smithi047 ceph-mon[39856]: pgmap v138: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:05:54.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:05:54 smithi149 ceph-mon[33230]: pgmap v138: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:05:54.667 INFO:teuthology.orchestra.run.smithi047.stderr:++ ceph auth get-key osd.0 2024-09-10T19:05:54.996 INFO:teuthology.orchestra.run.smithi047.stderr:+ NK=AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== 2024-09-10T19:05:54.996 INFO:teuthology.orchestra.run.smithi047.stderr:+ '[' AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== == AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== ']' 2024-09-10T19:05:54.996 INFO:teuthology.orchestra.run.smithi047.stderr:+ sleep 5 2024-09-10T19:05:55.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:05:55 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/3802213080' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:05:55.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:05:55 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/3802213080' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:05:55.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:05:55 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/3802213080' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:05:56.039 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:05:55 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: ::ffff:172.21.15.149 - - [10/Sep/2024:19:05:55] "GET /metrics HTTP/1.1" 200 219458 "" "Prometheus/2.43.0" 2024-09-10T19:05:56.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:05:56 smithi047 ceph-mon[29019]: pgmap v139: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:05:56.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:05:56 smithi047 ceph-mon[39856]: pgmap v139: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:05:56.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:05:56 smithi149 ceph-mon[33230]: pgmap v139: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:05:57.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:05:57 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:05:57.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:05:57 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:05:57.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:05:57 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:05:58.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:05:58 smithi047 ceph-mon[29019]: pgmap v140: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:05:58.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:05:58 smithi047 ceph-mon[39856]: pgmap v140: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:05:58.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:05:58 smithi149 ceph-mon[33230]: pgmap v140: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:05:58.593 INFO:journalctl@ceph.iscsi.iscsi.a.smithi149.stdout:Sep 10 19:05:58 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-iscsi-iscsi-a[60538]: debug there is no tcmu-runner data available 2024-09-10T19:05:59.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:05:59 smithi047 ceph-mon[29019]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:05:59.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:05:59 smithi047 ceph-mon[39856]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:05:59.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:05:59 smithi149 ceph-mon[33230]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:05:59.998 INFO:teuthology.orchestra.run.smithi047.stderr:++ ceph auth get-key osd.0 2024-09-10T19:06:00.327 INFO:teuthology.orchestra.run.smithi047.stderr:+ NK=AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== 2024-09-10T19:06:00.327 INFO:teuthology.orchestra.run.smithi047.stderr:+ '[' AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== == AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== ']' 2024-09-10T19:06:00.327 INFO:teuthology.orchestra.run.smithi047.stderr:+ sleep 5 2024-09-10T19:06:00.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:06:00 smithi047 ceph-mon[29019]: pgmap v141: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:06:00.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:06:00 smithi047 ceph-mon[39856]: pgmap v141: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:06:00.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:06:00 smithi149 ceph-mon[33230]: pgmap v141: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:06:01.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:06:01 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/1621163226' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:06:01.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:06:01 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/1621163226' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:06:01.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:06:01 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/1621163226' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:06:02.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:06:02 smithi149 ceph-mon[33230]: pgmap v142: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:06:02.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:06:02 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:06:02.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:06:02 smithi047 ceph-mon[29019]: pgmap v142: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:06:02.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:06:02 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:06:02.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:06:02 smithi047 ceph-mon[39856]: pgmap v142: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:06:02.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:06:02 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:06:03.295 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:06:02 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: ::ffff:172.21.15.149 - - [10/Sep/2024:19:06:02] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:06:03.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:06:03 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:06:03.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:06:03 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:06:03.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:06:03 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:06:03.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:06:03 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:06:03.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:06:03 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:06:03.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:06:03 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:06:03.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:06:03 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:06:03.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:06:03 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:06:03.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:06:03 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:06:04.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:06:04 smithi149 ceph-mon[33230]: pgmap v143: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:06:04.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:06:04 smithi047 ceph-mon[29019]: pgmap v143: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:06:04.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:06:04 smithi047 ceph-mon[39856]: pgmap v143: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:06:05.329 INFO:teuthology.orchestra.run.smithi047.stderr:++ ceph auth get-key osd.0 2024-09-10T19:06:05.661 INFO:teuthology.orchestra.run.smithi047.stderr:+ NK=AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== 2024-09-10T19:06:05.661 INFO:teuthology.orchestra.run.smithi047.stderr:+ '[' AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== == AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== ']' 2024-09-10T19:06:05.661 INFO:teuthology.orchestra.run.smithi047.stderr:+ sleep 5 2024-09-10T19:06:06.040 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:06:05 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: ::ffff:172.21.15.149 - - [10/Sep/2024:19:06:05] "GET /metrics HTTP/1.1" 200 219466 "" "Prometheus/2.43.0" 2024-09-10T19:06:06.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:06:06 smithi149 ceph-mon[33230]: pgmap v144: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:06:06.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:06:06 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/880138324' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:06:06.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:06:06 smithi047 ceph-mon[29019]: pgmap v144: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:06:06.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:06:06 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/880138324' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:06:06.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:06:06 smithi047 ceph-mon[39856]: pgmap v144: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:06:06.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:06:06 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/880138324' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:06:08.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:06:08 smithi149 ceph-mon[33230]: pgmap v145: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:06:08.593 INFO:journalctl@ceph.iscsi.iscsi.a.smithi149.stdout:Sep 10 19:06:08 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-iscsi-iscsi-a[60538]: debug there is no tcmu-runner data available 2024-09-10T19:06:08.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:06:08 smithi047 ceph-mon[29019]: pgmap v145: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:06:08.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:06:08 smithi047 ceph-mon[39856]: pgmap v145: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:06:09.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:06:09 smithi149 ceph-mon[33230]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:06:09.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:06:09 smithi047 ceph-mon[29019]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:06:09.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:06:09 smithi047 ceph-mon[39856]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:06:10.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:06:10 smithi149 ceph-mon[33230]: pgmap v146: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:06:10.663 INFO:teuthology.orchestra.run.smithi047.stderr:++ ceph auth get-key osd.0 2024-09-10T19:06:10.663 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:06:10 smithi047 ceph-mon[29019]: pgmap v146: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:06:10.664 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:06:10 smithi047 ceph-mon[39856]: pgmap v146: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:06:10.990 INFO:teuthology.orchestra.run.smithi047.stderr:+ NK=AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== 2024-09-10T19:06:10.991 INFO:teuthology.orchestra.run.smithi047.stderr:+ '[' AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== == AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== ']' 2024-09-10T19:06:10.991 INFO:teuthology.orchestra.run.smithi047.stderr:+ sleep 5 2024-09-10T19:06:11.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:06:11 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/2525459411' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:06:11.789 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:06:11 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/2525459411' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:06:11.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:06:11 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/2525459411' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:06:12.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:06:12 smithi149 ceph-mon[33230]: pgmap v147: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:06:12.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:06:12 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:06:12.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:06:12 smithi047 ceph-mon[29019]: pgmap v147: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:06:12.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:06:12 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:06:12.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:06:12 smithi047 ceph-mon[39856]: pgmap v147: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:06:12.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:06:12 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:06:13.343 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:06:12 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: ::ffff:172.21.15.149 - - [10/Sep/2024:19:06:12] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:06:14.596 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:06:14 smithi149 ceph-mon[33230]: pgmap v148: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:06:14.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:06:14 smithi047 ceph-mon[29019]: pgmap v148: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:06:14.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:06:14 smithi047 ceph-mon[39856]: pgmap v148: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:06:15.993 INFO:teuthology.orchestra.run.smithi047.stderr:++ ceph auth get-key osd.0 2024-09-10T19:06:15.994 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:06:15 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: ::ffff:172.21.15.149 - - [10/Sep/2024:19:06:15] "GET /metrics HTTP/1.1" 200 219465 "" "Prometheus/2.43.0" 2024-09-10T19:06:16.320 INFO:teuthology.orchestra.run.smithi047.stdout:rotating key for osd.1 2024-09-10T19:06:16.320 INFO:teuthology.orchestra.run.smithi047.stderr:+ NK=AQDMl+Bmo8gNMRAAc7CbHBlRDzWd1890/9HWWQ== 2024-09-10T19:06:16.320 INFO:teuthology.orchestra.run.smithi047.stderr:+ '[' AQColuBmK1XsDhAAIRmMYCDwqX+m39uBk1bAPQ== == AQDMl+Bmo8gNMRAAc7CbHBlRDzWd1890/9HWWQ== ']' 2024-09-10T19:06:16.320 INFO:teuthology.orchestra.run.smithi047.stderr:+ for f in osd.0 osd.1 osd.2 osd.3 osd.4 osd.5 osd.6 osd.7 mgr.y mgr.x 2024-09-10T19:06:16.320 INFO:teuthology.orchestra.run.smithi047.stderr:+ echo 'rotating key for osd.1' 2024-09-10T19:06:16.320 INFO:teuthology.orchestra.run.smithi047.stderr:++ ceph auth get-key osd.1 2024-09-10T19:06:16.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:06:16 smithi149 ceph-mon[33230]: pgmap v149: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:06:16.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:06:16 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/1351048151' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:06:16.638 INFO:teuthology.orchestra.run.smithi047.stderr:+ K=AQC+luBmaCcjOhAAxfRJvGYdE6OSclvv/wj62w== 2024-09-10T19:06:16.638 INFO:teuthology.orchestra.run.smithi047.stderr:+ NK=AQC+luBmaCcjOhAAxfRJvGYdE6OSclvv/wj62w== 2024-09-10T19:06:16.638 INFO:teuthology.orchestra.run.smithi047.stderr:+ ceph orch daemon rotate-key osd.1 2024-09-10T19:06:16.638 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:06:16 smithi047 ceph-mon[39856]: pgmap v149: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:06:16.638 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:06:16 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/1351048151' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:06:16.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:06:16 smithi047 ceph-mon[29019]: pgmap v149: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:06:16.639 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:06:16 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/1351048151' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:06:16.901 INFO:teuthology.orchestra.run.smithi047.stdout:Scheduled to rotate-key osd.1 on host 'smithi047' 2024-09-10T19:06:16.913 INFO:teuthology.orchestra.run.smithi047.stderr:+ '[' AQC+luBmaCcjOhAAxfRJvGYdE6OSclvv/wj62w== == AQC+luBmaCcjOhAAxfRJvGYdE6OSclvv/wj62w== ']' 2024-09-10T19:06:16.913 INFO:teuthology.orchestra.run.smithi047.stderr:+ sleep 5 2024-09-10T19:06:17.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:06:17 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/1506290064' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-10T19:06:17.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:06:17 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:06:17.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:06:17 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:06:17.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:06:17 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:06:17.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:06:17 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:06:17.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:06:17 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:06:17.594 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:06:17 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:06:17.594 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:06:17 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.1", "format": "json"}]: dispatch 2024-09-10T19:06:17.594 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:06:17 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.1", "format": "json"}]': finished 2024-09-10T19:06:17.594 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:06:17 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/1506290064' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-10T19:06:17.594 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:06:17 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:06:17.595 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:06:17 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:06:17.595 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:06:17 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:06:17.595 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:06:17 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:06:17.595 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:06:17 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:06:17.595 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:06:17 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:06:17.595 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:06:17 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.1", "format": "json"}]: dispatch 2024-09-10T19:06:17.595 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:06:17 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.1", "format": "json"}]': finished 2024-09-10T19:06:17.595 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:06:17 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/1506290064' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-10T19:06:17.595 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:06:17 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:06:17.595 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:06:17 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:06:17.596 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:06:17 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:06:17.596 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:06:17 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:06:17.596 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:06:17 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:06:17.596 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:06:17 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:06:17.596 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:06:17 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.1", "format": "json"}]: dispatch 2024-09-10T19:06:17.596 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:06:17 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.1", "format": "json"}]': finished 2024-09-10T19:06:18.593 INFO:journalctl@ceph.iscsi.iscsi.a.smithi149.stdout:Sep 10 19:06:18 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-iscsi-iscsi-a[60538]: debug there is no tcmu-runner data available 2024-09-10T19:06:18.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:06:18 smithi149 ceph-mon[33230]: from='client.24923 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:06:18.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:06:18 smithi149 ceph-mon[33230]: Schedule rotate-key daemon osd.1 2024-09-10T19:06:18.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:06:18 smithi149 ceph-mon[33230]: Rotating authentication key for osd.1 2024-09-10T19:06:18.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:06:18 smithi149 ceph-mon[33230]: Reconfiguring daemon osd.1 on smithi047 2024-09-10T19:06:18.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:06:18 smithi149 ceph-mon[33230]: pgmap v150: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:06:18.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:06:18 smithi047 ceph-mon[29019]: from='client.24923 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:06:18.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:06:18 smithi047 ceph-mon[29019]: Schedule rotate-key daemon osd.1 2024-09-10T19:06:18.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:06:18 smithi047 ceph-mon[29019]: Rotating authentication key for osd.1 2024-09-10T19:06:18.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:06:18 smithi047 ceph-mon[29019]: Reconfiguring daemon osd.1 on smithi047 2024-09-10T19:06:18.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:06:18 smithi047 ceph-mon[29019]: pgmap v150: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:06:18.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:06:18 smithi047 ceph-mon[39856]: from='client.24923 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:06:18.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:06:18 smithi047 ceph-mon[39856]: Schedule rotate-key daemon osd.1 2024-09-10T19:06:18.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:06:18 smithi047 ceph-mon[39856]: Rotating authentication key for osd.1 2024-09-10T19:06:18.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:06:18 smithi047 ceph-mon[39856]: Reconfiguring daemon osd.1 on smithi047 2024-09-10T19:06:18.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:06:18 smithi047 ceph-mon[39856]: pgmap v150: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:06:19.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:06:19 smithi149 ceph-mon[33230]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:06:19.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:06:19 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:06:19.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:06:19 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:06:19.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:06:19 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:06:19.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:06:19 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:06:19.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:06:19 smithi047 ceph-mon[29019]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:06:19.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:06:19 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:06:19.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:06:19 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:06:19.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:06:19 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:06:19.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:06:19 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:06:19.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:06:19 smithi047 ceph-mon[39856]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:06:19.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:06:19 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:06:19.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:06:19 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:06:19.791 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:06:19 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:06:19.791 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:06:19 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:06:20.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:06:20 smithi149 ceph-mon[33230]: pgmap v151: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:06:20.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:06:20 smithi047 ceph-mon[29019]: pgmap v151: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:06:20.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:06:20 smithi047 ceph-mon[39856]: pgmap v151: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:06:21.916 INFO:teuthology.orchestra.run.smithi047.stderr:++ ceph auth get-key osd.1 2024-09-10T19:06:22.245 INFO:teuthology.orchestra.run.smithi047.stderr:+ NK=AQC+luBmaCcjOhAAxfRJvGYdE6OSclvv/wj62w== 2024-09-10T19:06:22.245 INFO:teuthology.orchestra.run.smithi047.stderr:+ '[' AQC+luBmaCcjOhAAxfRJvGYdE6OSclvv/wj62w== == AQC+luBmaCcjOhAAxfRJvGYdE6OSclvv/wj62w== ']' 2024-09-10T19:06:22.245 INFO:teuthology.orchestra.run.smithi047.stderr:+ sleep 5 2024-09-10T19:06:22.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:06:22 smithi047 ceph-mon[29019]: pgmap v152: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:06:22.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:06:22 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/1527836431' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-10T19:06:22.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:06:22 smithi047 ceph-mon[39856]: pgmap v152: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:06:22.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:06:22 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/1527836431' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-10T19:06:22.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:06:22 smithi149 ceph-mon[33230]: pgmap v152: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:06:22.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:06:22 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/1527836431' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-10T19:06:23.343 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:06:22 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: ::ffff:172.21.15.149 - - [10/Sep/2024:19:06:22] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:06:24.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:06:24 smithi149 ceph-mon[33230]: pgmap v153: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:06:24.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:06:24 smithi047 ceph-mon[29019]: pgmap v153: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:06:24.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:06:24 smithi047 ceph-mon[39856]: pgmap v153: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:06:26.040 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:06:25 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: ::ffff:172.21.15.149 - - [10/Sep/2024:19:06:25] "GET /metrics HTTP/1.1" 200 219465 "" "Prometheus/2.43.0" 2024-09-10T19:06:26.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:06:26 smithi149 ceph-mon[33230]: pgmap v154: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:06:26.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:06:26 smithi047 ceph-mon[29019]: pgmap v154: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:06:26.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:06:26 smithi047 ceph-mon[39856]: pgmap v154: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:06:27.247 INFO:teuthology.orchestra.run.smithi047.stderr:++ ceph auth get-key osd.1 2024-09-10T19:06:27.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:06:27 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:06:27.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:06:27 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:06:27.582 INFO:teuthology.orchestra.run.smithi047.stderr:+ NK=AQC+luBmaCcjOhAAxfRJvGYdE6OSclvv/wj62w== 2024-09-10T19:06:27.582 INFO:teuthology.orchestra.run.smithi047.stderr:+ '[' AQC+luBmaCcjOhAAxfRJvGYdE6OSclvv/wj62w== == AQC+luBmaCcjOhAAxfRJvGYdE6OSclvv/wj62w== ']' 2024-09-10T19:06:27.582 INFO:teuthology.orchestra.run.smithi047.stderr:+ sleep 5 2024-09-10T19:06:27.592 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:06:27 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:06:28.593 INFO:journalctl@ceph.iscsi.iscsi.a.smithi149.stdout:Sep 10 19:06:28 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-iscsi-iscsi-a[60538]: debug there is no tcmu-runner data available 2024-09-10T19:06:28.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:06:28 smithi149 ceph-mon[33230]: pgmap v155: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:06:28.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:06:28 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/2768644436' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-10T19:06:28.789 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:06:28 smithi047 ceph-mon[39856]: pgmap v155: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:06:28.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:06:28 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/2768644436' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-10T19:06:28.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:06:28 smithi047 ceph-mon[29019]: pgmap v155: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:06:28.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:06:28 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/2768644436' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-10T19:06:29.789 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:06:29 smithi047 ceph-mon[39856]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:06:29.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:06:29 smithi047 ceph-mon[29019]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:06:29.843 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:06:29 smithi149 ceph-mon[33230]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:06:30.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:06:30 smithi047 ceph-mon[29019]: pgmap v156: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:06:30.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:06:30 smithi047 ceph-mon[39856]: pgmap v156: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:06:30.842 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:06:30 smithi149 ceph-mon[33230]: pgmap v156: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:06:32.584 INFO:teuthology.orchestra.run.smithi047.stderr:++ ceph auth get-key osd.1 2024-09-10T19:06:32.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:06:32 smithi047 ceph-mon[29019]: pgmap v157: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:06:32.789 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:06:32 smithi047 ceph-mon[39856]: pgmap v157: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:06:32.843 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:06:32 smithi149 ceph-mon[33230]: pgmap v157: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:06:32.910 INFO:teuthology.orchestra.run.smithi047.stderr:+ NK=AQC+luBmaCcjOhAAxfRJvGYdE6OSclvv/wj62w== 2024-09-10T19:06:32.910 INFO:teuthology.orchestra.run.smithi047.stderr:+ '[' AQC+luBmaCcjOhAAxfRJvGYdE6OSclvv/wj62w== == AQC+luBmaCcjOhAAxfRJvGYdE6OSclvv/wj62w== ']' 2024-09-10T19:06:32.910 INFO:teuthology.orchestra.run.smithi047.stderr:+ sleep 5 2024-09-10T19:06:33.343 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:06:32 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: ::ffff:172.21.15.149 - - [10/Sep/2024:19:06:32] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:06:33.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:06:33 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/1991067755' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-10T19:06:33.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:06:33 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/1991067755' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-10T19:06:33.843 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:06:33 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/1991067755' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-10T19:06:34.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:06:34 smithi047 ceph-mon[29019]: pgmap v158: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:06:34.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:06:34 smithi047 ceph-mon[39856]: pgmap v158: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:06:34.843 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:06:34 smithi149 ceph-mon[33230]: pgmap v158: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:06:35.708 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:06:35 smithi047 ceph-mon[29019]: pgmap v159: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:06:35.708 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:06:35 smithi047 ceph-mon[39856]: pgmap v159: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:06:35.843 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:06:35 smithi149 ceph-mon[33230]: pgmap v159: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:06:36.039 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:06:35 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: ::ffff:172.21.15.149 - - [10/Sep/2024:19:06:35] "GET /metrics HTTP/1.1" 200 219466 "" "Prometheus/2.43.0" 2024-09-10T19:06:37.912 INFO:teuthology.orchestra.run.smithi047.stderr:++ ceph auth get-key osd.1 2024-09-10T19:06:38.248 INFO:teuthology.orchestra.run.smithi047.stderr:+ NK=AQC+luBmaCcjOhAAxfRJvGYdE6OSclvv/wj62w== 2024-09-10T19:06:38.248 INFO:teuthology.orchestra.run.smithi047.stderr:+ '[' AQC+luBmaCcjOhAAxfRJvGYdE6OSclvv/wj62w== == AQC+luBmaCcjOhAAxfRJvGYdE6OSclvv/wj62w== ']' 2024-09-10T19:06:38.248 INFO:teuthology.orchestra.run.smithi047.stderr:+ sleep 5 2024-09-10T19:06:38.249 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:06:38 smithi047 ceph-mon[29019]: pgmap v160: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:06:38.249 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:06:38 smithi047 ceph-mon[39856]: pgmap v160: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:06:38.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:06:38 smithi149 ceph-mon[33230]: pgmap v160: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:06:38.593 INFO:journalctl@ceph.iscsi.iscsi.a.smithi149.stdout:Sep 10 19:06:38 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-iscsi-iscsi-a[60538]: debug there is no tcmu-runner data available 2024-09-10T19:06:39.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:06:39 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/3170177639' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-10T19:06:39.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:06:39 smithi047 ceph-mon[29019]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:06:39.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:06:39 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/3170177639' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-10T19:06:39.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:06:39 smithi047 ceph-mon[39856]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:06:39.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:06:39 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/3170177639' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-10T19:06:39.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:06:39 smithi149 ceph-mon[33230]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:06:40.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:06:40 smithi047 ceph-mon[29019]: pgmap v161: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:06:40.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:06:40 smithi047 ceph-mon[39856]: pgmap v161: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:06:40.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:06:40 smithi149 ceph-mon[33230]: pgmap v161: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:06:42.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:06:42 smithi047 ceph-mon[29019]: pgmap v162: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:06:42.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:06:42 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:06:42.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:06:42 smithi047 ceph-mon[39856]: pgmap v162: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:06:42.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:06:42 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:06:42.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:06:42 smithi149 ceph-mon[33230]: pgmap v162: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:06:42.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:06:42 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:06:43.250 INFO:teuthology.orchestra.run.smithi047.stderr:++ ceph auth get-key osd.1 2024-09-10T19:06:43.343 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:06:42 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: ::ffff:172.21.15.149 - - [10/Sep/2024:19:06:42] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:06:43.581 INFO:teuthology.orchestra.run.smithi047.stdout:rotating key for osd.2 2024-09-10T19:06:43.581 INFO:teuthology.orchestra.run.smithi047.stderr:+ NK=AQComOBm3VObOhAAbGHV5DoV2KFaj0tkBFTwnA== 2024-09-10T19:06:43.581 INFO:teuthology.orchestra.run.smithi047.stderr:+ '[' AQC+luBmaCcjOhAAxfRJvGYdE6OSclvv/wj62w== == AQComOBm3VObOhAAbGHV5DoV2KFaj0tkBFTwnA== ']' 2024-09-10T19:06:43.581 INFO:teuthology.orchestra.run.smithi047.stderr:+ for f in osd.0 osd.1 osd.2 osd.3 osd.4 osd.5 osd.6 osd.7 mgr.y mgr.x 2024-09-10T19:06:43.581 INFO:teuthology.orchestra.run.smithi047.stderr:+ echo 'rotating key for osd.2' 2024-09-10T19:06:43.581 INFO:teuthology.orchestra.run.smithi047.stderr:++ ceph auth get-key osd.2 2024-09-10T19:06:43.897 INFO:teuthology.orchestra.run.smithi047.stderr:+ K=AQDVluBm8F1QKhAAWW49arLhs1Q8kwtyt0KvEg== 2024-09-10T19:06:43.897 INFO:teuthology.orchestra.run.smithi047.stderr:+ NK=AQDVluBm8F1QKhAAWW49arLhs1Q8kwtyt0KvEg== 2024-09-10T19:06:43.897 INFO:teuthology.orchestra.run.smithi047.stderr:+ ceph orch daemon rotate-key osd.2 2024-09-10T19:06:44.162 INFO:teuthology.orchestra.run.smithi047.stdout:Scheduled to rotate-key osd.2 on host 'smithi047' 2024-09-10T19:06:44.173 INFO:teuthology.orchestra.run.smithi047.stderr:+ '[' AQDVluBm8F1QKhAAWW49arLhs1Q8kwtyt0KvEg== == AQDVluBm8F1QKhAAWW49arLhs1Q8kwtyt0KvEg== ']' 2024-09-10T19:06:44.174 INFO:teuthology.orchestra.run.smithi047.stderr:+ sleep 5 2024-09-10T19:06:44.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:06:44 smithi047 ceph-mon[29019]: pgmap v163: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:06:44.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:06:44 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/1776552680' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-10T19:06:44.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:06:44 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/1067862425' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-10T19:06:44.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:06:44 smithi047 ceph-mon[39856]: pgmap v163: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:06:44.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:06:44 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/1776552680' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-10T19:06:44.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:06:44 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/1067862425' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-10T19:06:44.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:06:44 smithi149 ceph-mon[33230]: pgmap v163: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:06:44.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:06:44 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/1776552680' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-10T19:06:44.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:06:44 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/1067862425' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-10T19:06:45.158 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:06:45 smithi047 ceph-mon[39856]: from='client.15072 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:06:45.158 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:06:45 smithi047 ceph-mon[39856]: Schedule rotate-key daemon osd.2 2024-09-10T19:06:45.158 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:06:45 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:06:45.158 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:06:45 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:06:45.158 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:06:45 smithi047 ceph-mon[29019]: from='client.15072 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:06:45.158 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:06:45 smithi047 ceph-mon[29019]: Schedule rotate-key daemon osd.2 2024-09-10T19:06:45.159 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:06:45 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:06:45.159 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:06:45 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:06:45.159 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:06:45 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:06:45.159 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:06:45 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:06:45.159 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:06:45 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:06:45.159 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:06:45 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:06:45.159 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:06:45 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:06:45.159 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:06:45 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:06:45.159 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:06:45 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.2", "format": "json"}]: dispatch 2024-09-10T19:06:45.159 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:06:45 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.2", "format": "json"}]': finished 2024-09-10T19:06:45.539 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:06:45 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:06:45.539 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:06:45 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:06:45.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:06:45 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:06:45.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:06:45 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:06:45.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:06:45 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:06:45.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:06:45 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:06:45.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:06:45 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.2", "format": "json"}]: dispatch 2024-09-10T19:06:45.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:06:45 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.2", "format": "json"}]': finished 2024-09-10T19:06:45.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:06:45 smithi149 ceph-mon[33230]: from='client.15072 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:06:45.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:06:45 smithi149 ceph-mon[33230]: Schedule rotate-key daemon osd.2 2024-09-10T19:06:45.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:06:45 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:06:45.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:06:45 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:06:45.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:06:45 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:06:45.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:06:45 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:06:45.594 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:06:45 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:06:45.594 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:06:45 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:06:45.594 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:06:45 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:06:45.594 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:06:45 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:06:45.594 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:06:45 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.2", "format": "json"}]: dispatch 2024-09-10T19:06:45.594 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:06:45 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.2", "format": "json"}]': finished 2024-09-10T19:06:46.039 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:06:45 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: ::ffff:172.21.15.149 - - [10/Sep/2024:19:06:45] "GET /metrics HTTP/1.1" 200 219470 "" "Prometheus/2.43.0" 2024-09-10T19:06:46.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:06:46 smithi047 ceph-mon[39856]: Rotating authentication key for osd.2 2024-09-10T19:06:46.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:06:46 smithi047 ceph-mon[39856]: Reconfiguring daemon osd.2 on smithi047 2024-09-10T19:06:46.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:06:46 smithi047 ceph-mon[39856]: pgmap v164: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:06:46.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:06:46 smithi047 ceph-mon[29019]: Rotating authentication key for osd.2 2024-09-10T19:06:46.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:06:46 smithi047 ceph-mon[29019]: Reconfiguring daemon osd.2 on smithi047 2024-09-10T19:06:46.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:06:46 smithi047 ceph-mon[29019]: pgmap v164: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:06:46.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:06:46 smithi149 ceph-mon[33230]: Rotating authentication key for osd.2 2024-09-10T19:06:46.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:06:46 smithi149 ceph-mon[33230]: Reconfiguring daemon osd.2 on smithi047 2024-09-10T19:06:46.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:06:46 smithi149 ceph-mon[33230]: pgmap v164: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:06:47.539 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:06:47 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:06:47.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:06:47 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:06:47.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:06:47 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:06:47.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:06:47 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:06:47.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:06:47 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:06:47.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:06:47 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:06:47.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:06:47 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:06:47.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:06:47 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:06:47.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:06:47 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:06:47.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:06:47 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:06:47.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:06:47 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:06:47.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:06:47 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:06:48.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:06:48 smithi047 ceph-mon[29019]: pgmap v165: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:06:48.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:06:48 smithi047 ceph-mon[39856]: pgmap v165: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:06:48.593 INFO:journalctl@ceph.iscsi.iscsi.a.smithi149.stdout:Sep 10 19:06:48 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-iscsi-iscsi-a[60538]: debug there is no tcmu-runner data available 2024-09-10T19:06:48.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:06:48 smithi149 ceph-mon[33230]: pgmap v165: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:06:49.177 INFO:teuthology.orchestra.run.smithi047.stderr:++ ceph auth get-key osd.2 2024-09-10T19:06:49.494 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:06:49 smithi149 ceph-mon[33230]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:06:49.505 INFO:teuthology.orchestra.run.smithi047.stderr:+ NK=AQDVluBm8F1QKhAAWW49arLhs1Q8kwtyt0KvEg== 2024-09-10T19:06:49.505 INFO:teuthology.orchestra.run.smithi047.stderr:+ '[' AQDVluBm8F1QKhAAWW49arLhs1Q8kwtyt0KvEg== == AQDVluBm8F1QKhAAWW49arLhs1Q8kwtyt0KvEg== ']' 2024-09-10T19:06:49.505 INFO:teuthology.orchestra.run.smithi047.stderr:+ sleep 5 2024-09-10T19:06:49.506 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:06:49 smithi047 ceph-mon[29019]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:06:49.506 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:06:49 smithi047 ceph-mon[39856]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:06:50.539 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:06:50 smithi047 ceph-mon[39856]: pgmap v166: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:06:50.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:06:50 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/674179169' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-10T19:06:50.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:06:50 smithi047 ceph-mon[29019]: pgmap v166: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:06:50.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:06:50 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/674179169' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-10T19:06:50.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:06:50 smithi149 ceph-mon[33230]: pgmap v166: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:06:50.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:06:50 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/674179169' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-10T19:06:52.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:06:52 smithi047 ceph-mon[29019]: pgmap v167: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:06:52.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:06:52 smithi047 ceph-mon[39856]: pgmap v167: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:06:52.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:06:52 smithi149 ceph-mon[33230]: pgmap v167: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:06:53.343 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:06:52 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: ::ffff:172.21.15.149 - - [10/Sep/2024:19:06:52] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:06:54.507 INFO:teuthology.orchestra.run.smithi047.stderr:++ ceph auth get-key osd.2 2024-09-10T19:06:54.508 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:06:54 smithi047 ceph-mon[29019]: pgmap v168: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:06:54.508 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:06:54 smithi047 ceph-mon[39856]: pgmap v168: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:06:54.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:06:54 smithi149 ceph-mon[33230]: pgmap v168: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:06:54.838 INFO:teuthology.orchestra.run.smithi047.stderr:+ NK=AQDVluBm8F1QKhAAWW49arLhs1Q8kwtyt0KvEg== 2024-09-10T19:06:54.838 INFO:teuthology.orchestra.run.smithi047.stderr:+ '[' AQDVluBm8F1QKhAAWW49arLhs1Q8kwtyt0KvEg== == AQDVluBm8F1QKhAAWW49arLhs1Q8kwtyt0KvEg== ']' 2024-09-10T19:06:54.838 INFO:teuthology.orchestra.run.smithi047.stderr:+ sleep 5 2024-09-10T19:06:55.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:06:55 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/304366138' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-10T19:06:55.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:06:55 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/304366138' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-10T19:06:55.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:06:55 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/304366138' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-10T19:06:56.040 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:06:55 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: ::ffff:172.21.15.149 - - [10/Sep/2024:19:06:55] "GET /metrics HTTP/1.1" 200 219470 "" "Prometheus/2.43.0" 2024-09-10T19:06:56.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:06:56 smithi047 ceph-mon[29019]: pgmap v169: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:06:56.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:06:56 smithi047 ceph-mon[39856]: pgmap v169: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:06:56.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:06:56 smithi149 ceph-mon[33230]: pgmap v169: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:06:57.539 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:06:57 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:06:57.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:06:57 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:06:57.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:06:57 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:06:58.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:06:58 smithi047 ceph-mon[29019]: pgmap v170: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:06:58.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:06:58 smithi047 ceph-mon[39856]: pgmap v170: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:06:58.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:06:58 smithi149 ceph-mon[33230]: pgmap v170: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:06:58.593 INFO:journalctl@ceph.iscsi.iscsi.a.smithi149.stdout:Sep 10 19:06:58 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-iscsi-iscsi-a[60538]: debug there is no tcmu-runner data available 2024-09-10T19:06:59.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:06:59 smithi047 ceph-mon[29019]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:06:59.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:06:59 smithi047 ceph-mon[39856]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:06:59.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:06:59 smithi149 ceph-mon[33230]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:06:59.840 INFO:teuthology.orchestra.run.smithi047.stderr:++ ceph auth get-key osd.2 2024-09-10T19:07:00.176 INFO:teuthology.orchestra.run.smithi047.stderr:+ NK=AQDVluBm8F1QKhAAWW49arLhs1Q8kwtyt0KvEg== 2024-09-10T19:07:00.176 INFO:teuthology.orchestra.run.smithi047.stderr:+ '[' AQDVluBm8F1QKhAAWW49arLhs1Q8kwtyt0KvEg== == AQDVluBm8F1QKhAAWW49arLhs1Q8kwtyt0KvEg== ']' 2024-09-10T19:07:00.177 INFO:teuthology.orchestra.run.smithi047.stderr:+ sleep 5 2024-09-10T19:07:00.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:00 smithi047 ceph-mon[29019]: pgmap v171: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:07:00.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:00 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/1946634465' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-10T19:07:00.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:00 smithi047 ceph-mon[39856]: pgmap v171: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:07:00.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:00 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/1946634465' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-10T19:07:00.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:00 smithi149 ceph-mon[33230]: pgmap v171: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:07:00.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:00 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/1946634465' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-10T19:07:02.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:01 smithi047 ceph-mon[29019]: pgmap v172: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:07:02.040 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:01 smithi047 ceph-mon[39856]: pgmap v172: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:07:02.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:01 smithi149 ceph-mon[33230]: pgmap v172: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:07:03.343 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:07:02 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: ::ffff:172.21.15.149 - - [10/Sep/2024:19:07:02] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:07:04.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:04 smithi047 ceph-mon[29019]: pgmap v173: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:07:04.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:04 smithi047 ceph-mon[39856]: pgmap v173: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:07:04.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:04 smithi149 ceph-mon[33230]: pgmap v173: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:07:05.178 INFO:teuthology.orchestra.run.smithi047.stderr:++ ceph auth get-key osd.2 2024-09-10T19:07:05.503 INFO:teuthology.orchestra.run.smithi047.stdout:rotating key for osd.3 2024-09-10T19:07:05.504 INFO:teuthology.orchestra.run.smithi047.stderr:+ NK=AQDEmOBmeBExIBAAQNVla7S2P1Fj//6iB1wjTA== 2024-09-10T19:07:05.504 INFO:teuthology.orchestra.run.smithi047.stderr:+ '[' AQDVluBm8F1QKhAAWW49arLhs1Q8kwtyt0KvEg== == AQDEmOBmeBExIBAAQNVla7S2P1Fj//6iB1wjTA== ']' 2024-09-10T19:07:05.504 INFO:teuthology.orchestra.run.smithi047.stderr:+ for f in osd.0 osd.1 osd.2 osd.3 osd.4 osd.5 osd.6 osd.7 mgr.y mgr.x 2024-09-10T19:07:05.504 INFO:teuthology.orchestra.run.smithi047.stderr:+ echo 'rotating key for osd.3' 2024-09-10T19:07:05.504 INFO:teuthology.orchestra.run.smithi047.stderr:++ ceph auth get-key osd.3 2024-09-10T19:07:05.789 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:07:05 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: ::ffff:172.21.15.149 - - [10/Sep/2024:19:07:05] "GET /metrics HTTP/1.1" 200 219490 "" "Prometheus/2.43.0" 2024-09-10T19:07:05.819 INFO:teuthology.orchestra.run.smithi047.stderr:+ K=AQDsluBmpBvQDxAAE0fxPHsk5STsDGOEGhvyLQ== 2024-09-10T19:07:05.819 INFO:teuthology.orchestra.run.smithi047.stderr:+ NK=AQDsluBmpBvQDxAAE0fxPHsk5STsDGOEGhvyLQ== 2024-09-10T19:07:05.819 INFO:teuthology.orchestra.run.smithi047.stderr:+ ceph orch daemon rotate-key osd.3 2024-09-10T19:07:06.088 INFO:teuthology.orchestra.run.smithi047.stdout:Scheduled to rotate-key osd.3 on host 'smithi047' 2024-09-10T19:07:06.100 INFO:teuthology.orchestra.run.smithi047.stderr:+ '[' AQDsluBmpBvQDxAAE0fxPHsk5STsDGOEGhvyLQ== == AQDsluBmpBvQDxAAE0fxPHsk5STsDGOEGhvyLQ== ']' 2024-09-10T19:07:06.101 INFO:teuthology.orchestra.run.smithi047.stderr:+ sleep 5 2024-09-10T19:07:06.380 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:06 smithi047 ceph-mon[29019]: pgmap v174: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:07:06.380 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:06 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/2726515036' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-10T19:07:06.380 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:06 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/4074597693' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-10T19:07:06.381 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:06 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:07:06.381 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:06 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:07:06.381 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:06 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:07:06.381 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:06 smithi047 ceph-mon[39856]: pgmap v174: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:07:06.381 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:06 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/2726515036' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-10T19:07:06.381 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:06 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/4074597693' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-10T19:07:06.381 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:06 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:07:06.381 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:06 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:07:06.381 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:06 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:07:06.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:06 smithi149 ceph-mon[33230]: pgmap v174: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:07:06.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:06 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/2726515036' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-10T19:07:06.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:06 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/4074597693' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-10T19:07:06.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:06 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:07:06.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:06 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:07:06.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:06 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:07:07.430 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:07 smithi047 ceph-mon[39856]: from='client.15120 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:07:07.430 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:07 smithi047 ceph-mon[39856]: Schedule rotate-key daemon osd.3 2024-09-10T19:07:07.430 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:07 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:07:07.430 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:07 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:07:07.430 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:07 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:07:07.430 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:07 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:07:07.430 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:07 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:07:07.430 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:07 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.3", "format": "json"}]: dispatch 2024-09-10T19:07:07.430 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:07 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.3", "format": "json"}]': finished 2024-09-10T19:07:07.431 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:07 smithi047 ceph-mon[29019]: from='client.15120 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:07:07.431 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:07 smithi047 ceph-mon[29019]: Schedule rotate-key daemon osd.3 2024-09-10T19:07:07.431 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:07 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:07:07.431 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:07 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:07:07.431 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:07 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:07:07.431 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:07 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:07:07.431 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:07 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:07:07.431 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:07 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.3", "format": "json"}]: dispatch 2024-09-10T19:07:07.431 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:07 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.3", "format": "json"}]': finished 2024-09-10T19:07:07.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:07 smithi149 ceph-mon[33230]: from='client.15120 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:07:07.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:07 smithi149 ceph-mon[33230]: Schedule rotate-key daemon osd.3 2024-09-10T19:07:07.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:07 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:07:07.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:07 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:07:07.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:07 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:07:07.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:07 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:07:07.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:07 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:07:07.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:07 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.3", "format": "json"}]: dispatch 2024-09-10T19:07:07.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:07 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.3", "format": "json"}]': finished 2024-09-10T19:07:08.437 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:08 smithi047 ceph-mon[29019]: Rotating authentication key for osd.3 2024-09-10T19:07:08.437 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:08 smithi047 ceph-mon[29019]: Reconfiguring daemon osd.3 on smithi047 2024-09-10T19:07:08.437 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:08 smithi047 ceph-mon[29019]: pgmap v175: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:07:08.437 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:08 smithi047 ceph-mon[39856]: Rotating authentication key for osd.3 2024-09-10T19:07:08.437 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:08 smithi047 ceph-mon[39856]: Reconfiguring daemon osd.3 on smithi047 2024-09-10T19:07:08.437 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:08 smithi047 ceph-mon[39856]: pgmap v175: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:07:08.593 INFO:journalctl@ceph.iscsi.iscsi.a.smithi149.stdout:Sep 10 19:07:08 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-iscsi-iscsi-a[60538]: debug there is no tcmu-runner data available 2024-09-10T19:07:08.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:08 smithi149 ceph-mon[33230]: Rotating authentication key for osd.3 2024-09-10T19:07:08.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:08 smithi149 ceph-mon[33230]: Reconfiguring daemon osd.3 on smithi047 2024-09-10T19:07:08.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:08 smithi149 ceph-mon[33230]: pgmap v175: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:07:09.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:09 smithi047 ceph-mon[29019]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:07:09.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:09 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:07:09.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:09 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:07:09.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:09 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:07:09.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:09 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:07:09.541 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:09 smithi047 ceph-mon[39856]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:07:09.541 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:09 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:07:09.541 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:09 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:07:09.541 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:09 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:07:09.541 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:09 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:07:09.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:09 smithi149 ceph-mon[33230]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:07:09.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:09 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:07:09.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:09 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:07:09.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:09 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:07:09.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:09 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:07:10.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:10 smithi047 ceph-mon[29019]: pgmap v176: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:07:10.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:10 smithi047 ceph-mon[39856]: pgmap v176: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:07:10.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:10 smithi149 ceph-mon[33230]: pgmap v176: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:07:11.103 INFO:teuthology.orchestra.run.smithi047.stderr:++ ceph auth get-key osd.3 2024-09-10T19:07:11.434 INFO:teuthology.orchestra.run.smithi047.stderr:+ NK=AQDsluBmpBvQDxAAE0fxPHsk5STsDGOEGhvyLQ== 2024-09-10T19:07:11.435 INFO:teuthology.orchestra.run.smithi047.stderr:+ '[' AQDsluBmpBvQDxAAE0fxPHsk5STsDGOEGhvyLQ== == AQDsluBmpBvQDxAAE0fxPHsk5STsDGOEGhvyLQ== ']' 2024-09-10T19:07:11.435 INFO:teuthology.orchestra.run.smithi047.stderr:+ sleep 5 2024-09-10T19:07:12.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:12 smithi047 ceph-mon[29019]: pgmap v177: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:07:12.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:12 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/2468070726' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-10T19:07:12.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:12 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:07:12.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:12 smithi047 ceph-mon[39856]: pgmap v177: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:07:12.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:12 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/2468070726' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-10T19:07:12.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:12 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:07:12.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:12 smithi149 ceph-mon[33230]: pgmap v177: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:07:12.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:12 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/2468070726' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-10T19:07:12.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:12 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:07:13.343 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:07:12 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: ::ffff:172.21.15.149 - - [10/Sep/2024:19:07:12] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:07:14.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:14 smithi047 ceph-mon[29019]: pgmap v178: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:07:14.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:14 smithi047 ceph-mon[39856]: pgmap v178: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:07:14.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:14 smithi149 ceph-mon[33230]: pgmap v178: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:07:16.039 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:07:15 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: ::ffff:172.21.15.149 - - [10/Sep/2024:19:07:15] "GET /metrics HTTP/1.1" 200 219493 "" "Prometheus/2.43.0" 2024-09-10T19:07:16.436 INFO:teuthology.orchestra.run.smithi047.stderr:++ ceph auth get-key osd.3 2024-09-10T19:07:16.437 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:16 smithi047 ceph-mon[29019]: pgmap v179: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:07:16.437 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:16 smithi047 ceph-mon[39856]: pgmap v179: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:07:16.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:16 smithi149 ceph-mon[33230]: pgmap v179: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:07:16.765 INFO:teuthology.orchestra.run.smithi047.stderr:+ NK=AQDsluBmpBvQDxAAE0fxPHsk5STsDGOEGhvyLQ== 2024-09-10T19:07:16.765 INFO:teuthology.orchestra.run.smithi047.stderr:+ '[' AQDsluBmpBvQDxAAE0fxPHsk5STsDGOEGhvyLQ== == AQDsluBmpBvQDxAAE0fxPHsk5STsDGOEGhvyLQ== ']' 2024-09-10T19:07:16.765 INFO:teuthology.orchestra.run.smithi047.stderr:+ sleep 5 2024-09-10T19:07:17.507 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:17 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/2744738922' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-10T19:07:17.507 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:17 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/2744738922' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-10T19:07:17.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:17 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/2744738922' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-10T19:07:18.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:18 smithi047 ceph-mon[29019]: pgmap v180: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:07:18.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:18 smithi047 ceph-mon[39856]: pgmap v180: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:07:18.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:18 smithi149 ceph-mon[33230]: pgmap v180: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:07:18.593 INFO:journalctl@ceph.iscsi.iscsi.a.smithi149.stdout:Sep 10 19:07:18 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-iscsi-iscsi-a[60538]: debug there is no tcmu-runner data available 2024-09-10T19:07:19.494 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:19 smithi149 ceph-mon[33230]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:07:19.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:19 smithi047 ceph-mon[29019]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:07:19.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:19 smithi047 ceph-mon[39856]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:07:20.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:20 smithi047 ceph-mon[29019]: pgmap v181: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:07:20.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:20 smithi047 ceph-mon[39856]: pgmap v181: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:07:20.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:20 smithi149 ceph-mon[33230]: pgmap v181: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:07:21.767 INFO:teuthology.orchestra.run.smithi047.stderr:++ ceph auth get-key osd.3 2024-09-10T19:07:22.095 INFO:teuthology.orchestra.run.smithi047.stderr:+ NK=AQDsluBmpBvQDxAAE0fxPHsk5STsDGOEGhvyLQ== 2024-09-10T19:07:22.096 INFO:teuthology.orchestra.run.smithi047.stderr:+ '[' AQDsluBmpBvQDxAAE0fxPHsk5STsDGOEGhvyLQ== == AQDsluBmpBvQDxAAE0fxPHsk5STsDGOEGhvyLQ== ']' 2024-09-10T19:07:22.096 INFO:teuthology.orchestra.run.smithi047.stderr:+ sleep 5 2024-09-10T19:07:22.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:22 smithi047 ceph-mon[29019]: pgmap v182: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:07:22.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:22 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/1707109268' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-10T19:07:22.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:22 smithi047 ceph-mon[39856]: pgmap v182: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:07:22.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:22 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/1707109268' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-10T19:07:22.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:22 smithi149 ceph-mon[33230]: pgmap v182: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:07:22.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:22 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/1707109268' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-10T19:07:23.343 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:07:22 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: ::ffff:172.21.15.149 - - [10/Sep/2024:19:07:22] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:07:24.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:24 smithi047 ceph-mon[39856]: pgmap v183: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:07:24.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:24 smithi047 ceph-mon[29019]: pgmap v183: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:07:24.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:24 smithi149 ceph-mon[33230]: pgmap v183: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:07:26.039 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:07:25 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: ::ffff:172.21.15.149 - - [10/Sep/2024:19:07:25] "GET /metrics HTTP/1.1" 200 219493 "" "Prometheus/2.43.0" 2024-09-10T19:07:26.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:26 smithi047 ceph-mon[29019]: pgmap v184: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:07:26.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:26 smithi047 ceph-mon[39856]: pgmap v184: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:07:26.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:26 smithi149 ceph-mon[33230]: pgmap v184: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:07:27.097 INFO:teuthology.orchestra.run.smithi047.stderr:++ ceph auth get-key osd.3 2024-09-10T19:07:27.429 INFO:teuthology.orchestra.run.smithi047.stdout:rotating key for osd.4 2024-09-10T19:07:27.429 INFO:teuthology.orchestra.run.smithi047.stderr:+ NK=AQDamOBm3xT9MxAA06N0y2bHrtaMqbgC4VDk/Q== 2024-09-10T19:07:27.430 INFO:teuthology.orchestra.run.smithi047.stderr:+ '[' AQDsluBmpBvQDxAAE0fxPHsk5STsDGOEGhvyLQ== == AQDamOBm3xT9MxAA06N0y2bHrtaMqbgC4VDk/Q== ']' 2024-09-10T19:07:27.430 INFO:teuthology.orchestra.run.smithi047.stderr:+ for f in osd.0 osd.1 osd.2 osd.3 osd.4 osd.5 osd.6 osd.7 mgr.y mgr.x 2024-09-10T19:07:27.430 INFO:teuthology.orchestra.run.smithi047.stderr:+ echo 'rotating key for osd.4' 2024-09-10T19:07:27.430 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:27 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:07:27.430 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:27 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:07:27.430 INFO:teuthology.orchestra.run.smithi047.stderr:++ ceph auth get-key osd.4 2024-09-10T19:07:27.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:27 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:07:27.745 INFO:teuthology.orchestra.run.smithi047.stderr:+ K=AQABl+BmWb32MxAA00nvrimyZUbnhws+LFELYQ== 2024-09-10T19:07:27.745 INFO:teuthology.orchestra.run.smithi047.stderr:+ NK=AQABl+BmWb32MxAA00nvrimyZUbnhws+LFELYQ== 2024-09-10T19:07:27.746 INFO:teuthology.orchestra.run.smithi047.stderr:+ ceph orch daemon rotate-key osd.4 2024-09-10T19:07:28.009 INFO:teuthology.orchestra.run.smithi047.stdout:Scheduled to rotate-key osd.4 on host 'smithi149' 2024-09-10T19:07:28.022 INFO:teuthology.orchestra.run.smithi047.stderr:+ '[' AQABl+BmWb32MxAA00nvrimyZUbnhws+LFELYQ== == AQABl+BmWb32MxAA00nvrimyZUbnhws+LFELYQ== ']' 2024-09-10T19:07:28.022 INFO:teuthology.orchestra.run.smithi047.stderr:+ sleep 5 2024-09-10T19:07:28.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:28 smithi047 ceph-mon[39856]: pgmap v185: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:07:28.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:28 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/1625780671' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-10T19:07:28.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:28 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/1935048592' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-10T19:07:28.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:28 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:07:28.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:28 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:07:28.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:28 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:07:28.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:28 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:07:28.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:28 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:07:28.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:28 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:07:28.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:28 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.4", "format": "json"}]: dispatch 2024-09-10T19:07:28.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:28 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.4", "format": "json"}]': finished 2024-09-10T19:07:28.291 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:28 smithi047 ceph-mon[29019]: pgmap v185: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:07:28.291 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:28 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/1625780671' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-10T19:07:28.291 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:28 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/1935048592' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-10T19:07:28.291 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:28 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:07:28.291 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:28 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:07:28.291 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:28 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:07:28.291 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:28 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:07:28.291 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:28 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:07:28.291 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:28 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:07:28.291 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:28 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.4", "format": "json"}]: dispatch 2024-09-10T19:07:28.292 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:28 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.4", "format": "json"}]': finished 2024-09-10T19:07:28.457 INFO:journalctl@ceph.iscsi.iscsi.a.smithi149.stdout:Sep 10 19:07:28 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-iscsi-iscsi-a[60538]: debug there is no tcmu-runner data available 2024-09-10T19:07:28.457 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:28 smithi149 ceph-mon[33230]: pgmap v185: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:07:28.457 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:28 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/1625780671' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-10T19:07:28.458 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:28 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/1935048592' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-10T19:07:28.458 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:28 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:07:28.458 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:28 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:07:28.458 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:28 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:07:28.458 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:28 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:07:28.458 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:28 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:07:28.458 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:28 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:07:28.458 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:28 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.4", "format": "json"}]: dispatch 2024-09-10T19:07:28.458 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:28 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.4", "format": "json"}]': finished 2024-09-10T19:07:29.533 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:29 smithi149 ceph-mon[33230]: from='client.15168 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:07:29.534 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:29 smithi149 ceph-mon[33230]: Schedule rotate-key daemon osd.4 2024-09-10T19:07:29.534 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:29 smithi149 ceph-mon[33230]: Rotating authentication key for osd.4 2024-09-10T19:07:29.534 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:29 smithi149 ceph-mon[33230]: Reconfiguring daemon osd.4 on smithi149 2024-09-10T19:07:29.534 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:29 smithi149 ceph-mon[33230]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:07:29.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:29 smithi047 ceph-mon[29019]: from='client.15168 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:07:29.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:29 smithi047 ceph-mon[29019]: Schedule rotate-key daemon osd.4 2024-09-10T19:07:29.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:29 smithi047 ceph-mon[29019]: Rotating authentication key for osd.4 2024-09-10T19:07:29.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:29 smithi047 ceph-mon[29019]: Reconfiguring daemon osd.4 on smithi149 2024-09-10T19:07:29.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:29 smithi047 ceph-mon[29019]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:07:29.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:29 smithi047 ceph-mon[39856]: from='client.15168 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:07:29.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:29 smithi047 ceph-mon[39856]: Schedule rotate-key daemon osd.4 2024-09-10T19:07:29.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:29 smithi047 ceph-mon[39856]: Rotating authentication key for osd.4 2024-09-10T19:07:29.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:29 smithi047 ceph-mon[39856]: Reconfiguring daemon osd.4 on smithi149 2024-09-10T19:07:29.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:29 smithi047 ceph-mon[39856]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:07:30.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:30 smithi047 ceph-mon[29019]: pgmap v186: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:07:30.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:30 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:07:30.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:30 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:07:30.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:30 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:07:30.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:30 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:07:30.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:30 smithi047 ceph-mon[39856]: pgmap v186: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:07:30.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:30 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:07:30.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:30 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:07:30.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:30 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:07:30.541 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:30 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:07:30.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:30 smithi149 ceph-mon[33230]: pgmap v186: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:07:30.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:30 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:07:30.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:30 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:07:30.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:30 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:07:30.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:30 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:07:32.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:32 smithi047 ceph-mon[29019]: pgmap v187: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:07:32.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:32 smithi047 ceph-mon[39856]: pgmap v187: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:07:32.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:32 smithi149 ceph-mon[33230]: pgmap v187: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:07:33.024 INFO:teuthology.orchestra.run.smithi047.stderr:++ ceph auth get-key osd.4 2024-09-10T19:07:33.344 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:07:32 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: ::ffff:172.21.15.149 - - [10/Sep/2024:19:07:32] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:07:33.357 INFO:teuthology.orchestra.run.smithi047.stderr:+ NK=AQABl+BmWb32MxAA00nvrimyZUbnhws+LFELYQ== 2024-09-10T19:07:33.357 INFO:teuthology.orchestra.run.smithi047.stderr:+ '[' AQABl+BmWb32MxAA00nvrimyZUbnhws+LFELYQ== == AQABl+BmWb32MxAA00nvrimyZUbnhws+LFELYQ== ']' 2024-09-10T19:07:33.357 INFO:teuthology.orchestra.run.smithi047.stderr:+ sleep 5 2024-09-10T19:07:34.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:34 smithi047 ceph-mon[29019]: pgmap v188: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:07:34.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:34 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/3093974756' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-10T19:07:34.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:34 smithi047 ceph-mon[39856]: pgmap v188: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:07:34.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:34 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/3093974756' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-10T19:07:34.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:34 smithi149 ceph-mon[33230]: pgmap v188: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:07:34.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:34 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/3093974756' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-10T19:07:36.039 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:07:35 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: ::ffff:172.21.15.149 - - [10/Sep/2024:19:07:35] "GET /metrics HTTP/1.1" 200 219494 "" "Prometheus/2.43.0" 2024-09-10T19:07:36.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:36 smithi047 ceph-mon[29019]: pgmap v189: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:07:36.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:36 smithi047 ceph-mon[39856]: pgmap v189: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:07:36.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:36 smithi149 ceph-mon[33230]: pgmap v189: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:07:38.360 INFO:teuthology.orchestra.run.smithi047.stderr:++ ceph auth get-key osd.4 2024-09-10T19:07:38.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:38 smithi047 ceph-mon[29019]: pgmap v190: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:07:38.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:38 smithi047 ceph-mon[39856]: pgmap v190: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:07:38.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:38 smithi149 ceph-mon[33230]: pgmap v190: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:07:38.593 INFO:journalctl@ceph.iscsi.iscsi.a.smithi149.stdout:Sep 10 19:07:38 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-iscsi-iscsi-a[60538]: debug there is no tcmu-runner data available 2024-09-10T19:07:38.675 INFO:teuthology.orchestra.run.smithi047.stderr:+ NK=AQABl+BmWb32MxAA00nvrimyZUbnhws+LFELYQ== 2024-09-10T19:07:38.675 INFO:teuthology.orchestra.run.smithi047.stderr:+ '[' AQABl+BmWb32MxAA00nvrimyZUbnhws+LFELYQ== == AQABl+BmWb32MxAA00nvrimyZUbnhws+LFELYQ== ']' 2024-09-10T19:07:38.675 INFO:teuthology.orchestra.run.smithi047.stderr:+ sleep 5 2024-09-10T19:07:39.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:39 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/4176292751' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-10T19:07:39.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:39 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/4176292751' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-10T19:07:39.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:39 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/4176292751' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-10T19:07:40.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:40 smithi047 ceph-mon[29019]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:07:40.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:40 smithi047 ceph-mon[29019]: pgmap v191: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:07:40.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:40 smithi047 ceph-mon[39856]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:07:40.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:40 smithi047 ceph-mon[39856]: pgmap v191: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:07:40.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:40 smithi149 ceph-mon[33230]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:07:40.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:40 smithi149 ceph-mon[33230]: pgmap v191: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:07:42.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:42 smithi047 ceph-mon[29019]: pgmap v192: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:07:42.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:42 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:07:42.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:42 smithi047 ceph-mon[39856]: pgmap v192: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:07:42.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:42 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:07:42.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:42 smithi149 ceph-mon[33230]: pgmap v192: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:07:42.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:42 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:07:43.343 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:07:42 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: ::ffff:172.21.15.149 - - [10/Sep/2024:19:07:42] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:07:43.677 INFO:teuthology.orchestra.run.smithi047.stderr:++ ceph auth get-key osd.4 2024-09-10T19:07:44.006 INFO:teuthology.orchestra.run.smithi047.stderr:+ NK=AQABl+BmWb32MxAA00nvrimyZUbnhws+LFELYQ== 2024-09-10T19:07:44.007 INFO:teuthology.orchestra.run.smithi047.stderr:+ '[' AQABl+BmWb32MxAA00nvrimyZUbnhws+LFELYQ== == AQABl+BmWb32MxAA00nvrimyZUbnhws+LFELYQ== ']' 2024-09-10T19:07:44.007 INFO:teuthology.orchestra.run.smithi047.stderr:+ sleep 5 2024-09-10T19:07:44.289 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:44 smithi047 ceph-mon[29019]: pgmap v193: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:07:44.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:44 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/3383199089' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-10T19:07:44.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:44 smithi047 ceph-mon[39856]: pgmap v193: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:07:44.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:44 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/3383199089' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-10T19:07:44.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:44 smithi149 ceph-mon[33230]: pgmap v193: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:07:44.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:44 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/3383199089' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-10T19:07:46.040 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:07:45 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: ::ffff:172.21.15.149 - - [10/Sep/2024:19:07:45] "GET /metrics HTTP/1.1" 200 219495 "" "Prometheus/2.43.0" 2024-09-10T19:07:46.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:46 smithi047 ceph-mon[29019]: pgmap v194: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:07:46.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:46 smithi047 ceph-mon[39856]: pgmap v194: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:07:46.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:46 smithi149 ceph-mon[33230]: pgmap v194: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:07:48.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:48 smithi047 ceph-mon[29019]: pgmap v195: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:07:48.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:48 smithi047 ceph-mon[39856]: pgmap v195: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:07:48.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:48 smithi149 ceph-mon[33230]: pgmap v195: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:07:48.593 INFO:journalctl@ceph.iscsi.iscsi.a.smithi149.stdout:Sep 10 19:07:48 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-iscsi-iscsi-a[60538]: debug there is no tcmu-runner data available 2024-09-10T19:07:49.008 INFO:teuthology.orchestra.run.smithi047.stderr:++ ceph auth get-key osd.4 2024-09-10T19:07:49.338 INFO:teuthology.orchestra.run.smithi047.stdout:rotating key for osd.5 2024-09-10T19:07:49.339 INFO:teuthology.orchestra.run.smithi047.stderr:+ NK=AQDwmOBm6Wt0BRAAVIdEVfBu7CgZvmx5wgVeGw== 2024-09-10T19:07:49.339 INFO:teuthology.orchestra.run.smithi047.stderr:+ '[' AQABl+BmWb32MxAA00nvrimyZUbnhws+LFELYQ== == AQDwmOBm6Wt0BRAAVIdEVfBu7CgZvmx5wgVeGw== ']' 2024-09-10T19:07:49.339 INFO:teuthology.orchestra.run.smithi047.stderr:+ for f in osd.0 osd.1 osd.2 osd.3 osd.4 osd.5 osd.6 osd.7 mgr.y mgr.x 2024-09-10T19:07:49.339 INFO:teuthology.orchestra.run.smithi047.stderr:+ echo 'rotating key for osd.5' 2024-09-10T19:07:49.339 INFO:teuthology.orchestra.run.smithi047.stderr:++ ceph auth get-key osd.5 2024-09-10T19:07:49.658 INFO:teuthology.orchestra.run.smithi047.stderr:+ K=AQAYl+BmO3cHEhAA2jN2brWtfPGWl8DjyEHq6A== 2024-09-10T19:07:49.658 INFO:teuthology.orchestra.run.smithi047.stderr:+ NK=AQAYl+BmO3cHEhAA2jN2brWtfPGWl8DjyEHq6A== 2024-09-10T19:07:49.658 INFO:teuthology.orchestra.run.smithi047.stderr:+ ceph orch daemon rotate-key osd.5 2024-09-10T19:07:49.924 INFO:teuthology.orchestra.run.smithi047.stdout:Scheduled to rotate-key osd.5 on host 'smithi149' 2024-09-10T19:07:49.925 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:49 smithi047 ceph-mon[29019]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:07:49.925 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:49 smithi047 ceph-mon[29019]: pgmap v196: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:07:49.925 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:49 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/3591109657' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-10T19:07:49.925 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:49 smithi047 ceph-mon[39856]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:07:49.925 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:49 smithi047 ceph-mon[39856]: pgmap v196: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:07:49.925 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:49 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/3591109657' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-10T19:07:49.938 INFO:teuthology.orchestra.run.smithi047.stderr:+ '[' AQAYl+BmO3cHEhAA2jN2brWtfPGWl8DjyEHq6A== == AQAYl+BmO3cHEhAA2jN2brWtfPGWl8DjyEHq6A== ']' 2024-09-10T19:07:49.938 INFO:teuthology.orchestra.run.smithi047.stderr:+ sleep 5 2024-09-10T19:07:50.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:49 smithi149 ceph-mon[33230]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:07:50.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:49 smithi149 ceph-mon[33230]: pgmap v196: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:07:50.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:49 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/3591109657' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-10T19:07:50.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:50 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/1487921727' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-10T19:07:51.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:50 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/1487921727' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-10T19:07:51.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:50 smithi047 ceph-mon[29019]: from='client.15216 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.5", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:07:51.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:50 smithi047 ceph-mon[29019]: Schedule rotate-key daemon osd.5 2024-09-10T19:07:51.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:50 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:07:51.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:50 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:07:51.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:50 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:07:51.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:50 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:07:51.041 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:50 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:07:51.041 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:50 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:07:51.041 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:50 smithi047 ceph-mon[29019]: Rotating authentication key for osd.5 2024-09-10T19:07:51.041 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:50 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.5", "format": "json"}]: dispatch 2024-09-10T19:07:51.041 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:50 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.5", "format": "json"}]': finished 2024-09-10T19:07:51.041 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:50 smithi047 ceph-mon[29019]: Reconfiguring daemon osd.5 on smithi149 2024-09-10T19:07:51.041 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:50 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/1487921727' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-10T19:07:51.041 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:50 smithi047 ceph-mon[39856]: from='client.15216 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.5", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:07:51.041 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:50 smithi047 ceph-mon[39856]: Schedule rotate-key daemon osd.5 2024-09-10T19:07:51.041 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:50 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:07:51.041 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:50 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:07:51.042 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:50 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:07:51.042 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:50 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:07:51.042 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:50 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:07:51.042 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:50 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:07:51.042 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:50 smithi047 ceph-mon[39856]: Rotating authentication key for osd.5 2024-09-10T19:07:51.042 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:50 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.5", "format": "json"}]: dispatch 2024-09-10T19:07:51.042 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:50 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.5", "format": "json"}]': finished 2024-09-10T19:07:51.042 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:50 smithi047 ceph-mon[39856]: Reconfiguring daemon osd.5 on smithi149 2024-09-10T19:07:51.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:50 smithi149 ceph-mon[33230]: from='client.15216 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.5", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:07:51.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:50 smithi149 ceph-mon[33230]: Schedule rotate-key daemon osd.5 2024-09-10T19:07:51.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:50 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:07:51.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:50 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:07:51.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:50 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:07:51.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:50 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:07:51.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:50 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:07:51.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:50 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:07:51.094 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:50 smithi149 ceph-mon[33230]: Rotating authentication key for osd.5 2024-09-10T19:07:51.094 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:50 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.5", "format": "json"}]: dispatch 2024-09-10T19:07:51.094 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:50 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.5", "format": "json"}]': finished 2024-09-10T19:07:51.094 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:50 smithi149 ceph-mon[33230]: Reconfiguring daemon osd.5 on smithi149 2024-09-10T19:07:52.039 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:51 smithi047 ceph-mon[29019]: pgmap v197: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:07:52.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:51 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:07:52.040 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:51 smithi047 ceph-mon[39856]: pgmap v197: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:07:52.040 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:51 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:07:52.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:51 smithi149 ceph-mon[33230]: pgmap v197: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:07:52.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:51 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:07:52.905 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:52 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:07:52.905 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:52 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:07:52.905 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:52 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:07:53.039 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:52 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:07:53.040 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:52 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:07:53.040 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:52 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:07:53.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:52 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:07:53.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:52 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:07:53.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:52 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:07:53.343 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:07:52 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: ::ffff:172.21.15.149 - - [10/Sep/2024:19:07:52] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:07:54.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:53 smithi047 ceph-mon[29019]: pgmap v198: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:07:54.040 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:53 smithi047 ceph-mon[39856]: pgmap v198: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:07:54.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:53 smithi149 ceph-mon[33230]: pgmap v198: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:07:54.940 INFO:teuthology.orchestra.run.smithi047.stderr:++ ceph auth get-key osd.5 2024-09-10T19:07:55.271 INFO:teuthology.orchestra.run.smithi047.stderr:+ NK=AQAYl+BmO3cHEhAA2jN2brWtfPGWl8DjyEHq6A== 2024-09-10T19:07:55.272 INFO:teuthology.orchestra.run.smithi047.stderr:+ '[' AQAYl+BmO3cHEhAA2jN2brWtfPGWl8DjyEHq6A== == AQAYl+BmO3cHEhAA2jN2brWtfPGWl8DjyEHq6A== ']' 2024-09-10T19:07:55.272 INFO:teuthology.orchestra.run.smithi047.stderr:+ sleep 5 2024-09-10T19:07:56.039 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:07:55 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: ::ffff:172.21.15.149 - - [10/Sep/2024:19:07:55] "GET /metrics HTTP/1.1" 200 219495 "" "Prometheus/2.43.0" 2024-09-10T19:07:56.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:56 smithi047 ceph-mon[29019]: pgmap v199: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:07:56.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:56 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/2782094981' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-10T19:07:56.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:56 smithi047 ceph-mon[39856]: pgmap v199: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:07:56.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:56 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/2782094981' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-10T19:07:56.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:56 smithi149 ceph-mon[33230]: pgmap v199: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:07:56.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:56 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/2782094981' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-10T19:07:57.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:57 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:07:57.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:57 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:07:57.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:57 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:07:58.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:07:58 smithi047 ceph-mon[29019]: pgmap v200: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:07:58.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:07:58 smithi047 ceph-mon[39856]: pgmap v200: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:07:58.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:07:58 smithi149 ceph-mon[33230]: pgmap v200: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:07:58.593 INFO:journalctl@ceph.iscsi.iscsi.a.smithi149.stdout:Sep 10 19:07:58 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-iscsi-iscsi-a[60538]: debug there is no tcmu-runner data available 2024-09-10T19:08:00.273 INFO:teuthology.orchestra.run.smithi047.stderr:++ ceph auth get-key osd.5 2024-09-10T19:08:00.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:00 smithi047 ceph-mon[29019]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:08:00.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:00 smithi047 ceph-mon[29019]: pgmap v201: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:08:00.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:00 smithi047 ceph-mon[39856]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:08:00.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:00 smithi047 ceph-mon[39856]: pgmap v201: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:08:00.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:00 smithi149 ceph-mon[33230]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:08:00.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:00 smithi149 ceph-mon[33230]: pgmap v201: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:08:00.600 INFO:teuthology.orchestra.run.smithi047.stderr:+ NK=AQAYl+BmO3cHEhAA2jN2brWtfPGWl8DjyEHq6A== 2024-09-10T19:08:00.600 INFO:teuthology.orchestra.run.smithi047.stderr:+ '[' AQAYl+BmO3cHEhAA2jN2brWtfPGWl8DjyEHq6A== == AQAYl+BmO3cHEhAA2jN2brWtfPGWl8DjyEHq6A== ']' 2024-09-10T19:08:00.600 INFO:teuthology.orchestra.run.smithi047.stderr:+ sleep 5 2024-09-10T19:08:01.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:01 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/1035843211' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-10T19:08:01.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:01 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/1035843211' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-10T19:08:01.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:01 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/1035843211' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-10T19:08:02.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:02 smithi047 ceph-mon[29019]: pgmap v202: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:08:02.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:02 smithi047 ceph-mon[39856]: pgmap v202: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:08:02.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:02 smithi149 ceph-mon[33230]: pgmap v202: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:08:03.343 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:08:02 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: ::ffff:172.21.15.149 - - [10/Sep/2024:19:08:02] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:08:04.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:04 smithi047 ceph-mon[29019]: pgmap v203: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:08:04.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:04 smithi047 ceph-mon[39856]: pgmap v203: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:08:04.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:04 smithi149 ceph-mon[33230]: pgmap v203: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:08:05.602 INFO:teuthology.orchestra.run.smithi047.stderr:++ ceph auth get-key osd.5 2024-09-10T19:08:05.935 INFO:teuthology.orchestra.run.smithi047.stderr:+ NK=AQAYl+BmO3cHEhAA2jN2brWtfPGWl8DjyEHq6A== 2024-09-10T19:08:05.936 INFO:teuthology.orchestra.run.smithi047.stderr:+ '[' AQAYl+BmO3cHEhAA2jN2brWtfPGWl8DjyEHq6A== == AQAYl+BmO3cHEhAA2jN2brWtfPGWl8DjyEHq6A== ']' 2024-09-10T19:08:05.936 INFO:teuthology.orchestra.run.smithi047.stderr:+ sleep 5 2024-09-10T19:08:05.936 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:08:05 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: ::ffff:172.21.15.149 - - [10/Sep/2024:19:08:05] "GET /metrics HTTP/1.1" 200 219503 "" "Prometheus/2.43.0" 2024-09-10T19:08:06.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:06 smithi047 ceph-mon[39856]: pgmap v204: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:08:06.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:06 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/2084117324' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-10T19:08:06.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:06 smithi047 ceph-mon[29019]: pgmap v204: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:08:06.541 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:06 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/2084117324' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-10T19:08:06.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:06 smithi149 ceph-mon[33230]: pgmap v204: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:08:06.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:06 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/2084117324' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-10T19:08:08.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:08 smithi047 ceph-mon[39856]: pgmap v205: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:08:08.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:08 smithi047 ceph-mon[29019]: pgmap v205: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:08:08.593 INFO:journalctl@ceph.iscsi.iscsi.a.smithi149.stdout:Sep 10 19:08:08 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-iscsi-iscsi-a[60538]: debug there is no tcmu-runner data available 2024-09-10T19:08:08.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:08 smithi149 ceph-mon[33230]: pgmap v205: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:08:10.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:10 smithi047 ceph-mon[29019]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:08:10.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:10 smithi047 ceph-mon[29019]: pgmap v206: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:08:10.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:10 smithi047 ceph-mon[39856]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:08:10.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:10 smithi047 ceph-mon[39856]: pgmap v206: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:08:10.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:10 smithi149 ceph-mon[33230]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:08:10.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:10 smithi149 ceph-mon[33230]: pgmap v206: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:08:10.937 INFO:teuthology.orchestra.run.smithi047.stderr:++ ceph auth get-key osd.5 2024-09-10T19:08:11.266 INFO:teuthology.orchestra.run.smithi047.stderr:+ NK=AQAGmeBmoEh8ABAACoDeGB8Qw/BDhHXatUSDzg== 2024-09-10T19:08:11.267 INFO:teuthology.orchestra.run.smithi047.stderr:+ '[' AQAYl+BmO3cHEhAA2jN2brWtfPGWl8DjyEHq6A== == AQAGmeBmoEh8ABAACoDeGB8Qw/BDhHXatUSDzg== ']' 2024-09-10T19:08:11.267 INFO:teuthology.orchestra.run.smithi047.stderr:+ for f in osd.0 osd.1 osd.2 osd.3 osd.4 osd.5 osd.6 osd.7 mgr.y mgr.x 2024-09-10T19:08:11.267 INFO:teuthology.orchestra.run.smithi047.stderr:+ echo 'rotating key for osd.6' 2024-09-10T19:08:11.267 INFO:teuthology.orchestra.run.smithi047.stdout:rotating key for osd.6 2024-09-10T19:08:11.267 INFO:teuthology.orchestra.run.smithi047.stderr:++ ceph auth get-key osd.6 2024-09-10T19:08:11.580 INFO:teuthology.orchestra.run.smithi047.stderr:+ K=AQAvl+BmMCD8ARAAglvuxwFuhWPq6u7qAogfxg== 2024-09-10T19:08:11.580 INFO:teuthology.orchestra.run.smithi047.stderr:+ NK=AQAvl+BmMCD8ARAAglvuxwFuhWPq6u7qAogfxg== 2024-09-10T19:08:11.580 INFO:teuthology.orchestra.run.smithi047.stderr:+ ceph orch daemon rotate-key osd.6 2024-09-10T19:08:11.846 INFO:teuthology.orchestra.run.smithi047.stdout:Scheduled to rotate-key osd.6 on host 'smithi149' 2024-09-10T19:08:11.858 INFO:teuthology.orchestra.run.smithi047.stderr:+ '[' AQAvl+BmMCD8ARAAglvuxwFuhWPq6u7qAogfxg== == AQAvl+BmMCD8ARAAglvuxwFuhWPq6u7qAogfxg== ']' 2024-09-10T19:08:11.858 INFO:teuthology.orchestra.run.smithi047.stderr:+ sleep 5 2024-09-10T19:08:12.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:12 smithi047 ceph-mon[29019]: pgmap v207: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:08:12.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:12 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/4102400611' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-10T19:08:12.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:12 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/3383218448' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-10T19:08:12.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:12 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:08:12.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:12 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:08:12.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:12 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:08:12.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:12 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:08:12.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:12 smithi047 ceph-mon[39856]: pgmap v207: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:08:12.541 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:12 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/4102400611' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-10T19:08:12.541 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:12 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/3383218448' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-10T19:08:12.541 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:12 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:08:12.541 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:12 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:08:12.541 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:12 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:08:12.541 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:12 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:08:12.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:12 smithi149 ceph-mon[33230]: pgmap v207: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:08:12.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:12 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/4102400611' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-10T19:08:12.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:12 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/3383218448' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-10T19:08:12.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:12 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:08:12.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:12 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:08:12.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:12 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:08:12.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:12 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:08:13.016 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:08:12 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: ::ffff:172.21.15.149 - - [10/Sep/2024:19:08:12] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:08:13.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:13 smithi149 ceph-mon[33230]: from='client.25169 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.6", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:08:13.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:13 smithi149 ceph-mon[33230]: Schedule rotate-key daemon osd.6 2024-09-10T19:08:13.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:13 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:08:13.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:13 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:08:13.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:13 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:08:13.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:13 smithi149 ceph-mon[33230]: Rotating authentication key for osd.6 2024-09-10T19:08:13.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:13 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.6", "format": "json"}]: dispatch 2024-09-10T19:08:13.344 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:13 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.6", "format": "json"}]': finished 2024-09-10T19:08:13.344 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:13 smithi149 ceph-mon[33230]: Reconfiguring daemon osd.6 on smithi149 2024-09-10T19:08:13.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:13 smithi047 ceph-mon[29019]: from='client.25169 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.6", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:08:13.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:13 smithi047 ceph-mon[29019]: Schedule rotate-key daemon osd.6 2024-09-10T19:08:13.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:13 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:08:13.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:13 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:08:13.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:13 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:08:13.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:13 smithi047 ceph-mon[29019]: Rotating authentication key for osd.6 2024-09-10T19:08:13.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:13 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.6", "format": "json"}]: dispatch 2024-09-10T19:08:13.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:13 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.6", "format": "json"}]': finished 2024-09-10T19:08:13.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:13 smithi047 ceph-mon[29019]: Reconfiguring daemon osd.6 on smithi149 2024-09-10T19:08:13.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:13 smithi047 ceph-mon[39856]: from='client.25169 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.6", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:08:13.541 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:13 smithi047 ceph-mon[39856]: Schedule rotate-key daemon osd.6 2024-09-10T19:08:13.541 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:13 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:08:13.541 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:13 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:08:13.541 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:13 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:08:13.541 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:13 smithi047 ceph-mon[39856]: Rotating authentication key for osd.6 2024-09-10T19:08:13.541 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:13 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.6", "format": "json"}]: dispatch 2024-09-10T19:08:13.541 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:13 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.6", "format": "json"}]': finished 2024-09-10T19:08:13.541 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:13 smithi047 ceph-mon[39856]: Reconfiguring daemon osd.6 on smithi149 2024-09-10T19:08:14.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:14 smithi149 ceph-mon[33230]: pgmap v208: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:08:14.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:14 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:08:14.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:14 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:08:14.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:14 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:08:14.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:14 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:08:14.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:14 smithi047 ceph-mon[29019]: pgmap v208: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:08:14.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:14 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:08:14.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:14 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:08:14.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:14 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:08:14.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:14 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:08:14.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:14 smithi047 ceph-mon[39856]: pgmap v208: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:08:14.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:14 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:08:14.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:14 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:08:14.791 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:14 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:08:14.791 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:14 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:08:16.039 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:08:15 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: ::ffff:172.21.15.149 - - [10/Sep/2024:19:08:15] "GET /metrics HTTP/1.1" 200 219500 "" "Prometheus/2.43.0" 2024-09-10T19:08:16.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:16 smithi149 ceph-mon[33230]: pgmap v209: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:08:16.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:16 smithi047 ceph-mon[29019]: pgmap v209: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:08:16.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:16 smithi047 ceph-mon[39856]: pgmap v209: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:08:16.860 INFO:teuthology.orchestra.run.smithi047.stderr:++ ceph auth get-key osd.6 2024-09-10T19:08:17.191 INFO:teuthology.orchestra.run.smithi047.stderr:+ NK=AQAvl+BmMCD8ARAAglvuxwFuhWPq6u7qAogfxg== 2024-09-10T19:08:17.191 INFO:teuthology.orchestra.run.smithi047.stderr:+ '[' AQAvl+BmMCD8ARAAglvuxwFuhWPq6u7qAogfxg== == AQAvl+BmMCD8ARAAglvuxwFuhWPq6u7qAogfxg== ']' 2024-09-10T19:08:17.192 INFO:teuthology.orchestra.run.smithi047.stderr:+ sleep 5 2024-09-10T19:08:17.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:17 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/2713726051' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-10T19:08:17.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:17 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/2713726051' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-10T19:08:17.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:17 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/2713726051' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-10T19:08:18.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:18 smithi149 ceph-mon[33230]: pgmap v210: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:08:18.594 INFO:journalctl@ceph.iscsi.iscsi.a.smithi149.stdout:Sep 10 19:08:18 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-iscsi-iscsi-a[60538]: debug there is no tcmu-runner data available 2024-09-10T19:08:18.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:18 smithi047 ceph-mon[39856]: pgmap v210: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:08:18.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:18 smithi047 ceph-mon[29019]: pgmap v210: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:08:20.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:20 smithi149 ceph-mon[33230]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:08:20.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:20 smithi149 ceph-mon[33230]: pgmap v211: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:08:20.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:20 smithi047 ceph-mon[29019]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:08:20.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:20 smithi047 ceph-mon[29019]: pgmap v211: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:08:20.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:20 smithi047 ceph-mon[39856]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:08:20.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:20 smithi047 ceph-mon[39856]: pgmap v211: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:08:22.193 INFO:teuthology.orchestra.run.smithi047.stderr:++ ceph auth get-key osd.6 2024-09-10T19:08:22.523 INFO:teuthology.orchestra.run.smithi047.stderr:+ NK=AQAvl+BmMCD8ARAAglvuxwFuhWPq6u7qAogfxg== 2024-09-10T19:08:22.523 INFO:teuthology.orchestra.run.smithi047.stderr:+ '[' AQAvl+BmMCD8ARAAglvuxwFuhWPq6u7qAogfxg== == AQAvl+BmMCD8ARAAglvuxwFuhWPq6u7qAogfxg== ']' 2024-09-10T19:08:22.523 INFO:teuthology.orchestra.run.smithi047.stderr:+ sleep 5 2024-09-10T19:08:22.524 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:22 smithi047 ceph-mon[29019]: pgmap v212: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:08:22.524 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:22 smithi047 ceph-mon[39856]: pgmap v212: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:08:22.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:22 smithi149 ceph-mon[33230]: pgmap v212: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:08:23.339 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:08:22 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: ::ffff:172.21.15.149 - - [10/Sep/2024:19:08:22] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:08:23.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:23 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/1834671177' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-10T19:08:23.789 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:23 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/1834671177' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-10T19:08:23.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:23 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/1834671177' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-10T19:08:24.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:24 smithi149 ceph-mon[33230]: pgmap v213: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:08:24.789 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:24 smithi047 ceph-mon[39856]: pgmap v213: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:08:24.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:24 smithi047 ceph-mon[29019]: pgmap v213: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:08:26.039 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:08:25 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: ::ffff:172.21.15.149 - - [10/Sep/2024:19:08:25] "GET /metrics HTTP/1.1" 200 219500 "" "Prometheus/2.43.0" 2024-09-10T19:08:26.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:26 smithi047 ceph-mon[29019]: pgmap v214: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:08:26.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:26 smithi047 ceph-mon[39856]: pgmap v214: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:08:26.843 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:26 smithi149 ceph-mon[33230]: pgmap v214: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:08:27.525 INFO:teuthology.orchestra.run.smithi047.stderr:++ ceph auth get-key osd.6 2024-09-10T19:08:27.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:27 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:08:27.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:27 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:08:27.843 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:27 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:08:27.856 INFO:teuthology.orchestra.run.smithi047.stderr:+ NK=AQAvl+BmMCD8ARAAglvuxwFuhWPq6u7qAogfxg== 2024-09-10T19:08:27.857 INFO:teuthology.orchestra.run.smithi047.stderr:+ '[' AQAvl+BmMCD8ARAAglvuxwFuhWPq6u7qAogfxg== == AQAvl+BmMCD8ARAAglvuxwFuhWPq6u7qAogfxg== ']' 2024-09-10T19:08:27.857 INFO:teuthology.orchestra.run.smithi047.stderr:+ sleep 5 2024-09-10T19:08:28.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:28 smithi047 ceph-mon[29019]: pgmap v215: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:08:28.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:28 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/4265770460' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-10T19:08:28.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:28 smithi047 ceph-mon[39856]: pgmap v215: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:08:28.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:28 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/4265770460' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-10T19:08:28.843 INFO:journalctl@ceph.iscsi.iscsi.a.smithi149.stdout:Sep 10 19:08:28 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-iscsi-iscsi-a[60538]: debug there is no tcmu-runner data available 2024-09-10T19:08:28.843 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:28 smithi149 ceph-mon[33230]: pgmap v215: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:08:28.843 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:28 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/4265770460' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-10T19:08:30.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:30 smithi047 ceph-mon[39856]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:08:30.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:30 smithi047 ceph-mon[39856]: pgmap v216: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:08:30.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:30 smithi047 ceph-mon[29019]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:08:30.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:30 smithi047 ceph-mon[29019]: pgmap v216: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:08:30.843 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:30 smithi149 ceph-mon[33230]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:08:30.843 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:30 smithi149 ceph-mon[33230]: pgmap v216: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:08:32.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:32 smithi149 ceph-mon[33230]: pgmap v217: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:08:32.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:32 smithi047 ceph-mon[29019]: pgmap v217: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:08:32.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:32 smithi047 ceph-mon[39856]: pgmap v217: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:08:32.858 INFO:teuthology.orchestra.run.smithi047.stderr:++ ceph auth get-key osd.6 2024-09-10T19:08:33.093 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:08:32 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: ::ffff:172.21.15.149 - - [10/Sep/2024:19:08:32] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:08:33.191 INFO:teuthology.orchestra.run.smithi047.stderr:+ NK=AQAcmeBmlcPEEhAA5mGByQrZQch0+/E5mDpBbQ== 2024-09-10T19:08:33.192 INFO:teuthology.orchestra.run.smithi047.stdout:rotating key for osd.7 2024-09-10T19:08:33.192 INFO:teuthology.orchestra.run.smithi047.stderr:+ '[' AQAvl+BmMCD8ARAAglvuxwFuhWPq6u7qAogfxg== == AQAcmeBmlcPEEhAA5mGByQrZQch0+/E5mDpBbQ== ']' 2024-09-10T19:08:33.192 INFO:teuthology.orchestra.run.smithi047.stderr:+ for f in osd.0 osd.1 osd.2 osd.3 osd.4 osd.5 osd.6 osd.7 mgr.y mgr.x 2024-09-10T19:08:33.192 INFO:teuthology.orchestra.run.smithi047.stderr:+ echo 'rotating key for osd.7' 2024-09-10T19:08:33.192 INFO:teuthology.orchestra.run.smithi047.stderr:++ ceph auth get-key osd.7 2024-09-10T19:08:33.515 INFO:teuthology.orchestra.run.smithi047.stderr:+ K=AQBFl+Bmfj9vMBAAKf2NVT+tUJmuHTmKFaGCfg== 2024-09-10T19:08:33.516 INFO:teuthology.orchestra.run.smithi047.stderr:+ NK=AQBFl+Bmfj9vMBAAKf2NVT+tUJmuHTmKFaGCfg== 2024-09-10T19:08:33.516 INFO:teuthology.orchestra.run.smithi047.stderr:+ ceph orch daemon rotate-key osd.7 2024-09-10T19:08:33.516 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:33 smithi047 ceph-mon[29019]: pgmap v218: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:08:33.516 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:33 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/681302416' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-10T19:08:33.516 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:33 smithi047 ceph-mon[39856]: pgmap v218: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:08:33.517 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:33 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/681302416' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-10T19:08:33.777 INFO:teuthology.orchestra.run.smithi047.stdout:Scheduled to rotate-key osd.7 on host 'smithi149' 2024-09-10T19:08:33.790 INFO:teuthology.orchestra.run.smithi047.stderr:+ '[' AQBFl+Bmfj9vMBAAKf2NVT+tUJmuHTmKFaGCfg== == AQBFl+Bmfj9vMBAAKf2NVT+tUJmuHTmKFaGCfg== ']' 2024-09-10T19:08:33.790 INFO:teuthology.orchestra.run.smithi047.stderr:+ sleep 5 2024-09-10T19:08:33.843 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:33 smithi149 ceph-mon[33230]: pgmap v218: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:08:33.843 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:33 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/681302416' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-10T19:08:34.572 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:34 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/1518896642' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-10T19:08:34.572 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:34 smithi149 ceph-mon[33230]: from='client.25217 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.7", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:08:34.572 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:34 smithi149 ceph-mon[33230]: Schedule rotate-key daemon osd.7 2024-09-10T19:08:34.572 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:34 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:08:34.572 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:34 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:08:34.572 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:34 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:08:34.572 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:34 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:08:34.572 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:34 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:08:34.572 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:34 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:08:34.572 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:34 smithi149 ceph-mon[33230]: Rotating authentication key for osd.7 2024-09-10T19:08:34.573 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:34 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.7", "format": "json"}]: dispatch 2024-09-10T19:08:34.573 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:34 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.7", "format": "json"}]': finished 2024-09-10T19:08:34.573 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:34 smithi149 ceph-mon[33230]: Reconfiguring daemon osd.7 on smithi149 2024-09-10T19:08:34.789 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:34 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/1518896642' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-10T19:08:34.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:34 smithi047 ceph-mon[39856]: from='client.25217 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.7", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:08:34.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:34 smithi047 ceph-mon[39856]: Schedule rotate-key daemon osd.7 2024-09-10T19:08:34.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:34 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:08:34.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:34 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:08:34.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:34 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:08:34.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:34 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:08:34.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:34 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:08:34.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:34 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:08:34.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:34 smithi047 ceph-mon[39856]: Rotating authentication key for osd.7 2024-09-10T19:08:34.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:34 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.7", "format": "json"}]: dispatch 2024-09-10T19:08:34.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:34 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.7", "format": "json"}]': finished 2024-09-10T19:08:34.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:34 smithi047 ceph-mon[39856]: Reconfiguring daemon osd.7 on smithi149 2024-09-10T19:08:34.791 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:34 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/1518896642' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-10T19:08:34.791 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:34 smithi047 ceph-mon[29019]: from='client.25217 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.7", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:08:34.791 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:34 smithi047 ceph-mon[29019]: Schedule rotate-key daemon osd.7 2024-09-10T19:08:34.791 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:34 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:08:34.791 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:34 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:08:34.791 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:34 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:08:34.791 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:34 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:08:34.791 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:34 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:08:34.791 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:34 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:08:34.791 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:34 smithi047 ceph-mon[29019]: Rotating authentication key for osd.7 2024-09-10T19:08:34.791 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:34 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.7", "format": "json"}]: dispatch 2024-09-10T19:08:34.792 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:34 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.7", "format": "json"}]': finished 2024-09-10T19:08:34.792 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:34 smithi047 ceph-mon[29019]: Reconfiguring daemon osd.7 on smithi149 2024-09-10T19:08:35.707 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:35 smithi047 ceph-mon[29019]: pgmap v219: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:08:35.708 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:35 smithi047 ceph-mon[39856]: pgmap v219: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:08:35.843 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:35 smithi149 ceph-mon[33230]: pgmap v219: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:08:36.040 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:08:35 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: ::ffff:172.21.15.149 - - [10/Sep/2024:19:08:35] "GET /metrics HTTP/1.1" 200 219503 "" "Prometheus/2.43.0" 2024-09-10T19:08:36.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:36 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:08:36.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:36 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:08:36.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:36 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:08:36.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:36 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:08:36.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:36 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:08:36.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:36 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:08:36.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:36 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:08:36.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:36 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:08:36.843 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:36 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:08:36.843 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:36 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:08:36.843 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:36 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:08:36.843 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:36 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:08:37.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:37 smithi047 ceph-mon[29019]: pgmap v220: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:08:37.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:37 smithi047 ceph-mon[39856]: pgmap v220: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:08:37.843 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:37 smithi149 ceph-mon[33230]: pgmap v220: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:08:38.793 INFO:teuthology.orchestra.run.smithi047.stderr:++ ceph auth get-key osd.7 2024-09-10T19:08:38.843 INFO:journalctl@ceph.iscsi.iscsi.a.smithi149.stdout:Sep 10 19:08:38 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-iscsi-iscsi-a[60538]: debug there is no tcmu-runner data available 2024-09-10T19:08:39.117 INFO:teuthology.orchestra.run.smithi047.stderr:+ NK=AQBFl+Bmfj9vMBAAKf2NVT+tUJmuHTmKFaGCfg== 2024-09-10T19:08:39.117 INFO:teuthology.orchestra.run.smithi047.stderr:+ '[' AQBFl+Bmfj9vMBAAKf2NVT+tUJmuHTmKFaGCfg== == AQBFl+Bmfj9vMBAAKf2NVT+tUJmuHTmKFaGCfg== ']' 2024-09-10T19:08:39.117 INFO:teuthology.orchestra.run.smithi047.stderr:+ sleep 5 2024-09-10T19:08:39.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:39 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/3126113579' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-10T19:08:39.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:39 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/3126113579' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-10T19:08:39.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:39 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/3126113579' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-10T19:08:40.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:40 smithi047 ceph-mon[29019]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:08:40.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:40 smithi047 ceph-mon[29019]: pgmap v221: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:08:40.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:40 smithi047 ceph-mon[39856]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:08:40.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:40 smithi047 ceph-mon[39856]: pgmap v221: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:08:40.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:40 smithi149 ceph-mon[33230]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:08:40.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:40 smithi149 ceph-mon[33230]: pgmap v221: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:08:42.539 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:42 smithi047 ceph-mon[39856]: pgmap v222: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:08:42.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:42 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:08:42.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:42 smithi047 ceph-mon[29019]: pgmap v222: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:08:42.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:42 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:08:42.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:42 smithi149 ceph-mon[33230]: pgmap v222: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:08:42.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:42 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:08:43.343 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:08:42 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: ::ffff:172.21.15.149 - - [10/Sep/2024:19:08:42] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:08:44.119 INFO:teuthology.orchestra.run.smithi047.stderr:++ ceph auth get-key osd.7 2024-09-10T19:08:44.450 INFO:teuthology.orchestra.run.smithi047.stderr:+ NK=AQBFl+Bmfj9vMBAAKf2NVT+tUJmuHTmKFaGCfg== 2024-09-10T19:08:44.450 INFO:teuthology.orchestra.run.smithi047.stderr:+ '[' AQBFl+Bmfj9vMBAAKf2NVT+tUJmuHTmKFaGCfg== == AQBFl+Bmfj9vMBAAKf2NVT+tUJmuHTmKFaGCfg== ']' 2024-09-10T19:08:44.450 INFO:teuthology.orchestra.run.smithi047.stderr:+ sleep 5 2024-09-10T19:08:44.450 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:44 smithi047 ceph-mon[29019]: pgmap v223: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:08:44.451 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:44 smithi047 ceph-mon[39856]: pgmap v223: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:08:44.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:44 smithi149 ceph-mon[33230]: pgmap v223: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:08:45.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:45 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/1362437399' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-10T19:08:45.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:45 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/1362437399' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-10T19:08:45.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:45 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/1362437399' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-10T19:08:46.039 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:08:45 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: ::ffff:172.21.15.149 - - [10/Sep/2024:19:08:45] "GET /metrics HTTP/1.1" 200 219501 "" "Prometheus/2.43.0" 2024-09-10T19:08:46.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:46 smithi047 ceph-mon[29019]: pgmap v224: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:08:46.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:46 smithi047 ceph-mon[39856]: pgmap v224: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:08:46.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:46 smithi149 ceph-mon[33230]: pgmap v224: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:08:48.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:48 smithi047 ceph-mon[29019]: pgmap v225: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:08:48.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:48 smithi047 ceph-mon[39856]: pgmap v225: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:08:48.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:48 smithi149 ceph-mon[33230]: pgmap v225: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:08:48.593 INFO:journalctl@ceph.iscsi.iscsi.a.smithi149.stdout:Sep 10 19:08:48 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-iscsi-iscsi-a[60538]: debug there is no tcmu-runner data available 2024-09-10T19:08:49.452 INFO:teuthology.orchestra.run.smithi047.stderr:++ ceph auth get-key osd.7 2024-09-10T19:08:49.779 INFO:teuthology.orchestra.run.smithi047.stderr:+ NK=AQBFl+Bmfj9vMBAAKf2NVT+tUJmuHTmKFaGCfg== 2024-09-10T19:08:49.779 INFO:teuthology.orchestra.run.smithi047.stderr:+ '[' AQBFl+Bmfj9vMBAAKf2NVT+tUJmuHTmKFaGCfg== == AQBFl+Bmfj9vMBAAKf2NVT+tUJmuHTmKFaGCfg== ']' 2024-09-10T19:08:49.779 INFO:teuthology.orchestra.run.smithi047.stderr:+ sleep 5 2024-09-10T19:08:50.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:50 smithi047 ceph-mon[29019]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:08:50.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:50 smithi047 ceph-mon[29019]: pgmap v226: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:08:50.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:50 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/3263542472' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-10T19:08:50.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:50 smithi047 ceph-mon[39856]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:08:50.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:50 smithi047 ceph-mon[39856]: pgmap v226: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:08:50.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:50 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/3263542472' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-10T19:08:50.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:50 smithi149 ceph-mon[33230]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:08:50.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:50 smithi149 ceph-mon[33230]: pgmap v226: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:08:50.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:50 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/3263542472' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-10T19:08:52.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:52 smithi047 ceph-mon[29019]: pgmap v227: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:08:52.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:52 smithi047 ceph-mon[39856]: pgmap v227: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:08:52.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:52 smithi149 ceph-mon[33230]: pgmap v227: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:08:53.343 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:08:52 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: ::ffff:172.21.15.149 - - [10/Sep/2024:19:08:52] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:08:53.792 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:53 smithi047 ceph-mon[29019]: pgmap v228: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:08:53.793 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:53 smithi047 ceph-mon[39856]: pgmap v228: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:08:53.843 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:53 smithi149 ceph-mon[33230]: pgmap v228: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:08:54.781 INFO:teuthology.orchestra.run.smithi047.stderr:++ ceph auth get-key osd.7 2024-09-10T19:08:55.112 INFO:teuthology.orchestra.run.smithi047.stdout:rotating key for mgr.y 2024-09-10T19:08:55.113 INFO:teuthology.orchestra.run.smithi047.stderr:+ NK=AQAxmeBmNzZ3MxAAHvQ8OMKQ0ck6xGh4ZxJh6A== 2024-09-10T19:08:55.113 INFO:teuthology.orchestra.run.smithi047.stderr:+ '[' AQBFl+Bmfj9vMBAAKf2NVT+tUJmuHTmKFaGCfg== == AQAxmeBmNzZ3MxAAHvQ8OMKQ0ck6xGh4ZxJh6A== ']' 2024-09-10T19:08:55.113 INFO:teuthology.orchestra.run.smithi047.stderr:+ for f in osd.0 osd.1 osd.2 osd.3 osd.4 osd.5 osd.6 osd.7 mgr.y mgr.x 2024-09-10T19:08:55.113 INFO:teuthology.orchestra.run.smithi047.stderr:+ echo 'rotating key for mgr.y' 2024-09-10T19:08:55.113 INFO:teuthology.orchestra.run.smithi047.stderr:++ ceph auth get-key mgr.y 2024-09-10T19:08:55.430 INFO:teuthology.orchestra.run.smithi047.stderr:+ K=AQAsluBm0dTEKxAA8EcraitHg2mj9uyD84vn1g== 2024-09-10T19:08:55.430 INFO:teuthology.orchestra.run.smithi047.stderr:+ NK=AQAsluBm0dTEKxAA8EcraitHg2mj9uyD84vn1g== 2024-09-10T19:08:55.430 INFO:teuthology.orchestra.run.smithi047.stderr:+ ceph orch daemon rotate-key mgr.y 2024-09-10T19:08:55.430 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:55 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/3112648649' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-10T19:08:55.431 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:55 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/3112648649' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-10T19:08:55.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:55 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/3112648649' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-10T19:08:55.696 INFO:teuthology.orchestra.run.smithi047.stdout:Scheduled to rotate-key mgr.y on host 'smithi047' 2024-09-10T19:08:55.708 INFO:teuthology.orchestra.run.smithi047.stderr:+ '[' AQAsluBm0dTEKxAA8EcraitHg2mj9uyD84vn1g== == AQAsluBm0dTEKxAA8EcraitHg2mj9uyD84vn1g== ']' 2024-09-10T19:08:55.708 INFO:teuthology.orchestra.run.smithi047.stderr:+ sleep 5 2024-09-10T19:08:56.039 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:08:55 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: ::ffff:172.21.15.149 - - [10/Sep/2024:19:08:55] "GET /metrics HTTP/1.1" 200 219501 "" "Prometheus/2.43.0" 2024-09-10T19:08:56.437 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:56 smithi047 ceph-mon[29019]: pgmap v229: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:08:56.437 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:56 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/4226024641' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.y"}]: dispatch 2024-09-10T19:08:56.437 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:56 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:08:56.437 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:56 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:08:56.437 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:56 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:08:56.437 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:56 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:08:56.437 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:56 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:08:56.437 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:56 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:08:56.437 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:56 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]: dispatch 2024-09-10T19:08:56.438 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:56 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]': finished 2024-09-10T19:08:56.438 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:56 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T19:08:56.438 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:56 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T19:08:56.438 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:56 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:08:56.438 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:56 smithi047 ceph-mon[39856]: pgmap v229: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:08:56.438 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:56 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/4226024641' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.y"}]: dispatch 2024-09-10T19:08:56.438 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:56 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:08:56.438 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:56 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:08:56.438 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:56 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:08:56.438 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:56 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:08:56.438 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:56 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:08:56.439 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:56 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:08:56.439 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:56 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]: dispatch 2024-09-10T19:08:56.439 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:56 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]': finished 2024-09-10T19:08:56.439 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:56 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T19:08:56.439 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:56 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T19:08:56.439 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:56 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:08:56.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:56 smithi149 ceph-mon[33230]: pgmap v229: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:08:56.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:56 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/4226024641' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.y"}]: dispatch 2024-09-10T19:08:56.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:56 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:08:56.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:56 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:08:56.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:56 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:08:56.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:56 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:08:56.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:56 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:08:56.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:56 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:08:56.594 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:56 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]: dispatch 2024-09-10T19:08:56.594 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:56 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]': finished 2024-09-10T19:08:56.594 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:56 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T19:08:56.594 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:56 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T19:08:56.594 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:56 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:08:57.495 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:57 smithi047 ceph-mon[29019]: from='client.15360 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "mgr.y", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:08:57.495 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:57 smithi047 ceph-mon[29019]: Schedule rotate-key daemon mgr.y 2024-09-10T19:08:57.495 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:57 smithi047 ceph-mon[29019]: Rotating authentication key for mgr.y 2024-09-10T19:08:57.495 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:57 smithi047 ceph-mon[29019]: Reconfiguring daemon mgr.y on smithi047 2024-09-10T19:08:57.495 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:57 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:08:57.496 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:57 smithi047 ceph-mon[39856]: from='client.15360 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "mgr.y", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:08:57.496 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:57 smithi047 ceph-mon[39856]: Schedule rotate-key daemon mgr.y 2024-09-10T19:08:57.496 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:57 smithi047 ceph-mon[39856]: Rotating authentication key for mgr.y 2024-09-10T19:08:57.496 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:57 smithi047 ceph-mon[39856]: Reconfiguring daemon mgr.y on smithi047 2024-09-10T19:08:57.496 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:57 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:08:57.513 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:57 smithi149 ceph-mon[33230]: from='client.15360 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "mgr.y", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:08:57.513 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:57 smithi149 ceph-mon[33230]: Schedule rotate-key daemon mgr.y 2024-09-10T19:08:57.513 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:57 smithi149 ceph-mon[33230]: Rotating authentication key for mgr.y 2024-09-10T19:08:57.513 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:57 smithi149 ceph-mon[33230]: Reconfiguring daemon mgr.y on smithi047 2024-09-10T19:08:57.513 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:57 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:08:57.771 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:08:57 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:08:57.493+0000 7f568111c640 -1 mgr.server reply reply (13) Permission denied access denied: does your client key have mgr caps? See http://docs.ceph.com/en/latest/mgr/administrator/#client-authentication 2024-09-10T19:08:57.771 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:08:57 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:08:57.495+0000 7f5673080640 -1 log_channel(cephadm) log [ERR] : Non-zero return from ['ceph', '-k', '/var/lib/ceph/mgr/ceph-y/keyring', '-n', 'mgr.y', 'tell', 'mgr.y', 'rotate-key', '-i', '-']: 2024-09-10T19:08:57.488+0000 7fdbb559a640 1 -- 172.21.15.47:0/3379014513 <== mon.2 v2:172.21.15.47:3301/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x7fdba0003710 con 0x7fdbb810ae80 2024-09-10T19:08:57.771 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:08:57 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:08:57.488+0000 7fdbb559a640 0 cephx client: could not set rotating key: decode_decrypt failed. error:bad magic in decode_decrypt, 8321543604190645795 != 18374858748799134293 2024-09-10T19:08:57.772 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:08:57 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:08:57.488+0000 7fdbbcf5b640 1 -- 172.21.15.47:0/3379014513 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdbb810ae80 msgr2=0x7fdbb810d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T19:08:57.772 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:08:57 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:08:57.488+0000 7fdbbcf5b640 1 --2- 172.21.15.47:0/3379014513 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdbb810ae80 0x7fdbb810d260 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fdbb0009d20 tx=0x7fdbb0004c50 comp rx=0 tx=0).stop 2024-09-10T19:08:57.772 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:08:57 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:08:57.488+0000 7fdbbcf5b640 1 -- 172.21.15.47:0/3379014513 shutdown_connections 2024-09-10T19:08:57.772 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:08:57 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:08:57.488+0000 7fdbbcf5b640 1 --2- 172.21.15.47:0/3379014513 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdbb810ae80 0x7fdbb810d260 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:08:57.772 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:08:57 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:08:57.488+0000 7fdbbcf5b640 1 --2- 172.21.15.47:0/3379014513 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdbb8065640 0x7fdbb810a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:08:57.772 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:08:57 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:08:57.488+0000 7fdbbcf5b640 1 --2- 172.21.15.47:0/3379014513 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdbb8064cf0 0x7fdbb8065100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:08:57.772 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:08:57 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:08:57.488+0000 7fdbbcf5b640 1 -- 172.21.15.47:0/3379014513 >> 172.21.15.47:0/3379014513 conn(0x7fdbb806aa40 msgr2=0x7fdbb806ce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T19:08:57.772 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:08:57 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:08:57.488+0000 7fdbbcf5b640 1 -- 172.21.15.47:0/3379014513 shutdown_connections 2024-09-10T19:08:57.772 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:08:57 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:08:57.488+0000 7fdbbcf5b640 1 -- 172.21.15.47:0/3379014513 wait complete. 2024-09-10T19:08:57.772 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:08:57 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:08:57.489+0000 7fdbbcf5b640 1 Processor -- start 2024-09-10T19:08:57.772 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:08:57 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:08:57.489+0000 7fdbbcf5b640 1 -- start start 2024-09-10T19:08:57.773 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:08:57 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:08:57.489+0000 7fdbbcf5b640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdbb8064cf0 0x7fdbb8106200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T19:08:57.773 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:08:57 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:08:57.489+0000 7fdbbcf5b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdbb8065640 0x7fdbb8104810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T19:08:57.773 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:08:57 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:08:57.489+0000 7fdbbcf5b640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdbb810ae80 0x7fdbb8104d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T19:08:57.773 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:08:57 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:08:57.489+0000 7fdbbcf5b640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb8105420 con 0x7fdbb8065640 2024-09-10T19:08:57.773 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:08:57 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:08:57.489+0000 7fdbbcf5b640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb8105590 con 0x7fdbb810ae80 2024-09-10T19:08:57.773 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:08:57 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:08:57.489+0000 7fdbbcf5b640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb811f400 con 0x7fdbb8064cf0 2024-09-10T19:08:57.773 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:08:57 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:08:57.489+0000 7fdbb659c640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdbb8064cf0 0x7fdbb8106200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T19:08:57.773 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:08:57 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:08:57.489+0000 7fdbb5d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdbb8065640 0x7fdbb8104810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T19:08:57.773 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:08:57 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:08:57.489+0000 7fdbb659c640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdbb8064cf0 0x7fdbb8106200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:42454/0 (socket says 172.21.15.47:42454) 2024-09-10T19:08:57.773 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:08:57 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:08:57.489+0000 7fdbb659c640 1 -- 172.21.15.47:0/3301252953 learned_addr learned my addr 172.21.15.47:0/3301252953 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-10T19:08:57.773 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:08:57 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:08:57.489+0000 7fdbb6d9d640 1 --2- 172.21.15.47:0/3301252953 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdbb810ae80 0x7fdbb8104d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T19:08:57.773 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:08:57 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:08:57.490+0000 7fdbb659c640 1 -- 172.21.15.47:0/3301252953 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdbb810ae80 msgr2=0x7fdbb8104d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T19:08:57.774 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:08:57 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:08:57.490+0000 7fdbb659c640 1 --2- 172.21.15.47:0/3301252953 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdbb810ae80 0x7fdbb8104d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:08:57.774 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:08:57 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:08:57.490+0000 7fdbb659c640 1 -- 172.21.15.47:0/3301252953 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdbb8065640 msgr2=0x7fdbb8104810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T19:08:57.774 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:08:57 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:08:57.490+0000 7fdbb659c640 1 --2- 172.21.15.47:0/3301252953 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdbb8065640 0x7fdbb8104810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:08:57.774 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:08:57 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:08:57.490+0000 7fdbb659c640 1 -- 172.21.15.47:0/3301252953 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdba8005040 con 0x7fdbb8064cf0 2024-09-10T19:08:57.774 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:08:57 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:08:57.490+0000 7fdbb5d9b640 1 --2- 172.21.15.47:0/3301252953 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdbb8065640 0x7fdbb8104810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T19:08:57.774 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:08:57 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:08:57.490+0000 7fdbb659c640 1 --2- 172.21.15.47:0/3301252953 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdbb8064cf0 0x7fdbb8106200 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fdba80094d0 tx=0x7fdba8009da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T19:08:57.774 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:08:57 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:08:57.490+0000 7fdba6ffd640 1 -- 172.21.15.47:0/3301252953 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba800e930 con 0x7fdbb8064cf0 2024-09-10T19:08:57.774 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:08:57 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:08:57.490+0000 7fdba6ffd640 1 -- 172.21.15.47:0/3301252953 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- auth(proto 2 2 bytes epoch 0) v1 -- 0x7fdba0003260 con 0x7fdbb8064cf0 2024-09-10T19:08:57.774 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:08:57 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:08:57.490+0000 7fdba6ffd640 1 -- 172.21.15.47:0/3301252953 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(38 keys) v1 ==== 1752+0+0 (secure 0 0 0) 0x7fdba800f040 con 0x7fdbb8064cf0 2024-09-10T19:08:57.774 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:08:57 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:08:57.490+0000 7fdbbcf5b640 1 -- 172.21.15.47:0/3301252953 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbb00098b0 con 0x7fdbb8064cf0 2024-09-10T19:08:57.774 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:08:57 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:08:57.490+0000 7fdba6ffd640 1 -- 172.21.15.47:0/3301252953 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba8012bb0 con 0x7fdbb8064cf0 2024-09-10T19:08:57.774 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:08:57 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:08:57.490+0000 7fdbbcf5b640 1 -- 172.21.15.47:0/3301252953 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbb811f9c0 con 0x7fdbb8064cf0 2024-09-10T19:08:57.774 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:08:57 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:08:57.490+0000 7fdba6ffd640 1 -- 172.21.15.47:0/3301252953 <== mon.2 v2:172.21.15.47:3301/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x7fdbb811f9c0 con 0x7fdbb8064cf0 2024-09-10T19:08:57.775 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:08:57 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:08:57.490+0000 7fdba6ffd640 0 cephx client: could not set rotating key: decode_decrypt failed. error:bad magic in decode_decrypt, 8321543604190645795 != 18374858748799134293 2024-09-10T19:08:57.775 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:08:57 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:08:57.492+0000 7fdba6ffd640 1 -- 172.21.15.47:0/3301252953 <== mon.2 v2:172.21.15.47:3301/0 5 ==== mgrmap(e 22) v1 ==== 90802+0+0 (secure 0 0 0) 0x7fdba8023020 con 0x7fdbb8064cf0 2024-09-10T19:08:57.775 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:08:57 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:08:57.492+0000 7fdba6ffd640 1 --2- 172.21.15.47:0/3301252953 >> v2:172.21.15.47:6800/743694085 conn(0x7fdba006f210 0x7fdba00716c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T19:08:57.775 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:08:57 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:08:57.492+0000 7fdba6ffd640 1 -- 172.21.15.47:0/3301252953 --> v2:172.21.15.47:6800/743694085 -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdba0071d90 con 0x7fdba006f210 2024-09-10T19:08:57.775 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:08:57 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:08:57.492+0000 7fdba6ffd640 1 -- 172.21.15.47:0/3301252953 <== mon.2 v2:172.21.15.47:3301/0 6 ==== osd_map(64..64 src has 1..64) v4 ==== 6284+0+0 (secure 0 0 0) 0x7fdba808ecf0 con 0x7fdbb8064cf0 2024-09-10T19:08:57.775 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:08:57 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:08:57.492+0000 7fdbb5d9b640 1 --2- 172.21.15.47:0/3301252953 >> v2:172.21.15.47:6800/743694085 conn(0x7fdba006f210 0x7fdba00716c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T19:08:57.775 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:08:57 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:08:57.493+0000 7fdbb5d9b640 1 --2- 172.21.15.47:0/3301252953 >> v2:172.21.15.47:6800/743694085 conn(0x7fdba006f210 0x7fdba00716c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fdbb810d690 tx=0x7fdbac00b040 comp rx=0 tx=0).ready entity=mgr.14562 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T19:08:57.775 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:08:57 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:08:57.493+0000 7fdba6ffd640 1 -- 172.21.15.47:0/3301252953 <== mgr.14562 v2:172.21.15.47:6800/743694085 1 ==== command_reply(tid 0: -13 access denied: does your client key have mgr caps? See http://docs.ceph.com/en/latest/mgr/administrator/#client-authentication) v1 ==== 134+0+0 (secure 0 0 0) 0x7fdba0071d90 con 0x7fdba006f210 2024-09-10T19:08:57.776 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:08:57 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: Error EACCES: problem getting command descriptions from mgr.y 2024-09-10T19:08:57.776 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:08:57 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:08:57.776 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:08:57 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:08:57.506+0000 7f5697b2b640 -1 mgr handle_mgr_map I was active but no longer am 2024-09-10T19:08:57.776 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:08:57 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: ignoring --setuser ceph since I am not root 2024-09-10T19:08:57.776 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:08:57 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: ignoring --setgroup ceph since I am not root 2024-09-10T19:08:57.776 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:08:57 smithi047 ceph-mgr[29248]: -- 172.21.15.47:0/2043113498 <== mon.2 v2:172.21.15.47:3301/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x556be096e340 con 0x556be097a800 2024-09-10T19:08:57.776 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:08:57 smithi047 ceph-mgr[29248]: cephx client: could not set rotating key: decode_decrypt failed. error:bad magic in decode_decrypt, 8321543604190645795 != 18374858748799134293 2024-09-10T19:08:57.776 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:08:57 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:08:57.645+0000 7f8de86abec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T19:08:57.776 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:08:57 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:08:57.701+0000 7f8de86abec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T19:08:57.777 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:08:57 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:08:57.768+0000 7f8de86abec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T19:08:57.843 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:08:57 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: [10/Sep/2024:19:08:57] ENGINE Bus STOPPING 2024-09-10T19:08:58.039 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:08:57 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:08:57.899+0000 7f8de86abec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T19:08:58.168 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:08:57 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: [10/Sep/2024:19:08:57] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-10T19:08:58.169 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:08:57 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: [10/Sep/2024:19:08:57] ENGINE Bus STOPPED 2024-09-10T19:08:58.169 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:08:57 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: [10/Sep/2024:19:08:57] ENGINE Bus STARTING 2024-09-10T19:08:58.169 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:08:58 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: [10/Sep/2024:19:08:58] ENGINE Serving on http://:::9283 2024-09-10T19:08:58.169 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:08:58 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: [10/Sep/2024:19:08:58] ENGINE Bus STARTED 2024-09-10T19:08:58.322 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:08:58 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:08:58.045+0000 7f8de86abec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T19:08:58.323 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:08:58 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:08:58.320+0000 7f8de86abec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T19:08:58.323 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:58 smithi047 ceph-mon[29019]: pgmap v230: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:08:58.323 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:58 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:08:58.323 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:58 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:08:58.324 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:58 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "mgr fail", "who": "y"}]: dispatch 2024-09-10T19:08:58.324 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:58 smithi047 ceph-mon[29019]: Activating manager daemon x 2024-09-10T19:08:58.324 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:58 smithi047 ceph-mon[29019]: osdmap e65: 8 total, 8 up, 8 in 2024-09-10T19:08:58.324 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:58 smithi047 ceph-mon[29019]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd='[{"prefix": "mgr fail", "who": "y"}]': finished 2024-09-10T19:08:58.324 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:58 smithi047 ceph-mon[29019]: mgrmap e23: x(active, starting, since 0.00711956s) 2024-09-10T19:08:58.324 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:58 smithi047 ceph-mon[29019]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T19:08:58.324 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:58 smithi047 ceph-mon[29019]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T19:08:58.324 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:58 smithi047 ceph-mon[29019]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T19:08:58.324 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:58 smithi047 ceph-mon[29019]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-10T19:08:58.324 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:58 smithi047 ceph-mon[29019]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T19:08:58.324 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:58 smithi047 ceph-mon[29019]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T19:08:58.324 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:58 smithi047 ceph-mon[29019]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T19:08:58.325 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:58 smithi047 ceph-mon[29019]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T19:08:58.325 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:58 smithi047 ceph-mon[29019]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T19:08:58.325 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:58 smithi047 ceph-mon[29019]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T19:08:58.325 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:58 smithi047 ceph-mon[29019]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T19:08:58.325 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:58 smithi047 ceph-mon[29019]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T19:08:58.325 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:58 smithi047 ceph-mon[29019]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T19:08:58.325 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:58 smithi047 ceph-mon[29019]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T19:08:58.325 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:58 smithi047 ceph-mon[29019]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T19:08:58.325 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:58 smithi047 ceph-mon[29019]: Manager daemon x is now available 2024-09-10T19:08:58.325 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:58 smithi047 ceph-mon[29019]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:08:58.325 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:58 smithi047 ceph-mon[29019]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:08:58.325 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:58 smithi047 ceph-mon[29019]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-10T19:08:58.326 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:58 smithi047 ceph-mon[29019]: from='mgr.24448 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-10T19:08:58.326 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:58 smithi047 ceph-mon[29019]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-10T19:08:58.326 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:58 smithi047 ceph-mon[29019]: from='mgr.24448 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-10T19:08:58.326 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:58 smithi047 ceph-mon[39856]: pgmap v230: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:08:58.326 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:58 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:08:58.326 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:58 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:08:58.327 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:58 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "mgr fail", "who": "y"}]: dispatch 2024-09-10T19:08:58.327 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:58 smithi047 ceph-mon[39856]: Activating manager daemon x 2024-09-10T19:08:58.327 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:58 smithi047 ceph-mon[39856]: osdmap e65: 8 total, 8 up, 8 in 2024-09-10T19:08:58.327 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:58 smithi047 ceph-mon[39856]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd='[{"prefix": "mgr fail", "who": "y"}]': finished 2024-09-10T19:08:58.327 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:58 smithi047 ceph-mon[39856]: mgrmap e23: x(active, starting, since 0.00711956s) 2024-09-10T19:08:58.327 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:58 smithi047 ceph-mon[39856]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T19:08:58.327 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:58 smithi047 ceph-mon[39856]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T19:08:58.327 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:58 smithi047 ceph-mon[39856]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T19:08:58.327 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:58 smithi047 ceph-mon[39856]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-10T19:08:58.327 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:58 smithi047 ceph-mon[39856]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T19:08:58.327 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:58 smithi047 ceph-mon[39856]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T19:08:58.327 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:58 smithi047 ceph-mon[39856]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T19:08:58.327 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:58 smithi047 ceph-mon[39856]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T19:08:58.328 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:58 smithi047 ceph-mon[39856]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T19:08:58.328 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:58 smithi047 ceph-mon[39856]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T19:08:58.328 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:58 smithi047 ceph-mon[39856]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T19:08:58.328 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:58 smithi047 ceph-mon[39856]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T19:08:58.328 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:58 smithi047 ceph-mon[39856]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T19:08:58.328 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:58 smithi047 ceph-mon[39856]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T19:08:58.328 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:58 smithi047 ceph-mon[39856]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T19:08:58.328 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:58 smithi047 ceph-mon[39856]: Manager daemon x is now available 2024-09-10T19:08:58.329 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:58 smithi047 ceph-mon[39856]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:08:58.329 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:58 smithi047 ceph-mon[39856]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:08:58.329 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:58 smithi047 ceph-mon[39856]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-10T19:08:58.329 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:58 smithi047 ceph-mon[39856]: from='mgr.24448 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-10T19:08:58.329 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:58 smithi047 ceph-mon[39856]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-10T19:08:58.329 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:58 smithi047 ceph-mon[39856]: from='mgr.24448 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-10T19:08:58.521 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:58 smithi149 ceph-mon[33230]: pgmap v230: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:08:58.521 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:58 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:08:58.521 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:58 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' 2024-09-10T19:08:58.521 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:58 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd=[{"prefix": "mgr fail", "who": "y"}]: dispatch 2024-09-10T19:08:58.521 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:58 smithi149 ceph-mon[33230]: Activating manager daemon x 2024-09-10T19:08:58.521 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:58 smithi149 ceph-mon[33230]: osdmap e65: 8 total, 8 up, 8 in 2024-09-10T19:08:58.521 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:58 smithi149 ceph-mon[33230]: from='mgr.14562 172.21.15.47:0/3468450227' entity='mgr.y' cmd='[{"prefix": "mgr fail", "who": "y"}]': finished 2024-09-10T19:08:58.521 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:58 smithi149 ceph-mon[33230]: mgrmap e23: x(active, starting, since 0.00711956s) 2024-09-10T19:08:58.521 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:58 smithi149 ceph-mon[33230]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T19:08:58.522 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:58 smithi149 ceph-mon[33230]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T19:08:58.522 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:58 smithi149 ceph-mon[33230]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T19:08:58.522 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:58 smithi149 ceph-mon[33230]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-10T19:08:58.522 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:58 smithi149 ceph-mon[33230]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T19:08:58.522 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:58 smithi149 ceph-mon[33230]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T19:08:58.522 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:58 smithi149 ceph-mon[33230]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T19:08:58.522 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:58 smithi149 ceph-mon[33230]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T19:08:58.522 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:58 smithi149 ceph-mon[33230]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T19:08:58.523 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:58 smithi149 ceph-mon[33230]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T19:08:58.523 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:58 smithi149 ceph-mon[33230]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T19:08:58.523 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:58 smithi149 ceph-mon[33230]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T19:08:58.523 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:58 smithi149 ceph-mon[33230]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T19:08:58.523 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:58 smithi149 ceph-mon[33230]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T19:08:58.523 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:58 smithi149 ceph-mon[33230]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T19:08:58.523 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:58 smithi149 ceph-mon[33230]: Manager daemon x is now available 2024-09-10T19:08:58.523 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:58 smithi149 ceph-mon[33230]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:08:58.523 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:58 smithi149 ceph-mon[33230]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:08:58.523 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:58 smithi149 ceph-mon[33230]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-10T19:08:58.523 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:58 smithi149 ceph-mon[33230]: from='mgr.24448 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-10T19:08:58.524 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:58 smithi149 ceph-mon[33230]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-10T19:08:58.524 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:58 smithi149 ceph-mon[33230]: from='mgr.24448 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-10T19:08:58.576 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:08:58 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:08:58.386+0000 7f8de86abec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T19:08:58.839 INFO:journalctl@ceph.iscsi.iscsi.a.smithi149.stdout:Sep 10 19:08:58 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-iscsi-iscsi-a[60538]: debug there is no tcmu-runner data available 2024-09-10T19:08:58.965 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:08:58 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:08:58.574+0000 7f8de86abec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T19:08:59.509 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:08:59 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:08:59.115+0000 7f0084d4c640 -1 log_channel(cephadm) log [ERR] : [10/Sep/2024:19:08:59] ENGINE Error in HTTPServer.serve 2024-09-10T19:08:59.509 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:08:59 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: Traceback (most recent call last): 2024-09-10T19:08:59.510 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:08:59 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T19:08:59.510 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:08:59 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: self._connections.run(self.expiration_interval) 2024-09-10T19:08:59.510 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:08:59 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T19:08:59.510 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:08:59 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: self._run(expiration_interval) 2024-09-10T19:08:59.510 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:08:59 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T19:08:59.510 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:08:59 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T19:08:59.510 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:08:59 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T19:08:59.510 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:08:59 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T19:08:59.510 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:08:59 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T19:08:59.510 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:08:59 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: s = self.context.wrap_socket( 2024-09-10T19:08:59.510 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:08:59 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T19:08:59.511 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:08:59 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: return self.sslsocket_class._create( 2024-09-10T19:08:59.511 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:08:59 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T19:08:59.511 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:08:59 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: self.do_handshake() 2024-09-10T19:08:59.511 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:08:59 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T19:08:59.511 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:08:59 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: self._sslobj.do_handshake() 2024-09-10T19:08:59.511 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:08:59 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T19:08:59.511 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:08:59 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:08:59.789 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:08:59 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:08:59.497+0000 7f8de86abec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T19:08:59.790 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:08:59 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:08:59.577+0000 7f8de86abec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T19:08:59.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:59 smithi047 ceph-mon[39856]: mgrmap e24: x(active, since 1.00806s) 2024-09-10T19:08:59.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:59 smithi047 ceph-mon[39856]: [10/Sep/2024:19:08:58] ENGINE Bus STARTING 2024-09-10T19:08:59.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:59 smithi047 ceph-mon[39856]: [10/Sep/2024:19:08:59] ENGINE Serving on https://172.21.15.149:7150 2024-09-10T19:08:59.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:59 smithi047 ceph-mon[39856]: [10/Sep/2024:19:08:59] ENGINE Bus STARTED 2024-09-10T19:08:59.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:59 smithi047 ceph-mon[39856]: [10/Sep/2024:19:08:59] ENGINE Error in HTTPServer.serve 2024-09-10T19:08:59.790 INFO:journalctl@ceph.mon.c.smithi047.stdout: Traceback (most recent call last): 2024-09-10T19:08:59.790 INFO:journalctl@ceph.mon.c.smithi047.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T19:08:59.790 INFO:journalctl@ceph.mon.c.smithi047.stdout: self._connections.run(self.expiration_interval) 2024-09-10T19:08:59.790 INFO:journalctl@ceph.mon.c.smithi047.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T19:08:59.791 INFO:journalctl@ceph.mon.c.smithi047.stdout: self._run(expiration_interval) 2024-09-10T19:08:59.791 INFO:journalctl@ceph.mon.c.smithi047.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T19:08:59.791 INFO:journalctl@ceph.mon.c.smithi047.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T19:08:59.791 INFO:journalctl@ceph.mon.c.smithi047.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T19:08:59.791 INFO:journalctl@ceph.mon.c.smithi047.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T19:08:59.791 INFO:journalctl@ceph.mon.c.smithi047.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T19:08:59.791 INFO:journalctl@ceph.mon.c.smithi047.stdout: s = self.context.wrap_socket( 2024-09-10T19:08:59.791 INFO:journalctl@ceph.mon.c.smithi047.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T19:08:59.791 INFO:journalctl@ceph.mon.c.smithi047.stdout: return self.sslsocket_class._create( 2024-09-10T19:08:59.791 INFO:journalctl@ceph.mon.c.smithi047.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T19:08:59.791 INFO:journalctl@ceph.mon.c.smithi047.stdout: self.do_handshake() 2024-09-10T19:08:59.791 INFO:journalctl@ceph.mon.c.smithi047.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T19:08:59.791 INFO:journalctl@ceph.mon.c.smithi047.stdout: self._sslobj.do_handshake() 2024-09-10T19:08:59.791 INFO:journalctl@ceph.mon.c.smithi047.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T19:08:59.792 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:08:59 smithi047 ceph-mon[39856]: from='mgr.24448 ' entity='mgr.x' 2024-09-10T19:08:59.792 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:59 smithi047 ceph-mon[29019]: mgrmap e24: x(active, since 1.00806s) 2024-09-10T19:08:59.792 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:59 smithi047 ceph-mon[29019]: [10/Sep/2024:19:08:58] ENGINE Bus STARTING 2024-09-10T19:08:59.792 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:59 smithi047 ceph-mon[29019]: [10/Sep/2024:19:08:59] ENGINE Serving on https://172.21.15.149:7150 2024-09-10T19:08:59.792 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:59 smithi047 ceph-mon[29019]: [10/Sep/2024:19:08:59] ENGINE Bus STARTED 2024-09-10T19:08:59.792 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:59 smithi047 ceph-mon[29019]: [10/Sep/2024:19:08:59] ENGINE Error in HTTPServer.serve 2024-09-10T19:08:59.792 INFO:journalctl@ceph.mon.a.smithi047.stdout: Traceback (most recent call last): 2024-09-10T19:08:59.792 INFO:journalctl@ceph.mon.a.smithi047.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T19:08:59.792 INFO:journalctl@ceph.mon.a.smithi047.stdout: self._connections.run(self.expiration_interval) 2024-09-10T19:08:59.792 INFO:journalctl@ceph.mon.a.smithi047.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T19:08:59.792 INFO:journalctl@ceph.mon.a.smithi047.stdout: self._run(expiration_interval) 2024-09-10T19:08:59.793 INFO:journalctl@ceph.mon.a.smithi047.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T19:08:59.793 INFO:journalctl@ceph.mon.a.smithi047.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T19:08:59.793 INFO:journalctl@ceph.mon.a.smithi047.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T19:08:59.793 INFO:journalctl@ceph.mon.a.smithi047.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T19:08:59.793 INFO:journalctl@ceph.mon.a.smithi047.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T19:08:59.793 INFO:journalctl@ceph.mon.a.smithi047.stdout: s = self.context.wrap_socket( 2024-09-10T19:08:59.793 INFO:journalctl@ceph.mon.a.smithi047.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T19:08:59.793 INFO:journalctl@ceph.mon.a.smithi047.stdout: return self.sslsocket_class._create( 2024-09-10T19:08:59.793 INFO:journalctl@ceph.mon.a.smithi047.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T19:08:59.793 INFO:journalctl@ceph.mon.a.smithi047.stdout: self.do_handshake() 2024-09-10T19:08:59.793 INFO:journalctl@ceph.mon.a.smithi047.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T19:08:59.793 INFO:journalctl@ceph.mon.a.smithi047.stdout: self._sslobj.do_handshake() 2024-09-10T19:08:59.793 INFO:journalctl@ceph.mon.a.smithi047.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T19:08:59.794 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:08:59 smithi047 ceph-mon[29019]: from='mgr.24448 ' entity='mgr.x' 2024-09-10T19:08:59.843 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:59 smithi149 ceph-mon[33230]: mgrmap e24: x(active, since 1.00806s) 2024-09-10T19:08:59.843 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:59 smithi149 ceph-mon[33230]: [10/Sep/2024:19:08:58] ENGINE Bus STARTING 2024-09-10T19:08:59.843 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:59 smithi149 ceph-mon[33230]: [10/Sep/2024:19:08:59] ENGINE Serving on https://172.21.15.149:7150 2024-09-10T19:08:59.843 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:59 smithi149 ceph-mon[33230]: [10/Sep/2024:19:08:59] ENGINE Bus STARTED 2024-09-10T19:08:59.843 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:59 smithi149 ceph-mon[33230]: [10/Sep/2024:19:08:59] ENGINE Error in HTTPServer.serve 2024-09-10T19:08:59.843 INFO:journalctl@ceph.mon.b.smithi149.stdout: Traceback (most recent call last): 2024-09-10T19:08:59.843 INFO:journalctl@ceph.mon.b.smithi149.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T19:08:59.843 INFO:journalctl@ceph.mon.b.smithi149.stdout: self._connections.run(self.expiration_interval) 2024-09-10T19:08:59.843 INFO:journalctl@ceph.mon.b.smithi149.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T19:08:59.844 INFO:journalctl@ceph.mon.b.smithi149.stdout: self._run(expiration_interval) 2024-09-10T19:08:59.844 INFO:journalctl@ceph.mon.b.smithi149.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T19:08:59.844 INFO:journalctl@ceph.mon.b.smithi149.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T19:08:59.844 INFO:journalctl@ceph.mon.b.smithi149.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T19:08:59.844 INFO:journalctl@ceph.mon.b.smithi149.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T19:08:59.844 INFO:journalctl@ceph.mon.b.smithi149.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T19:08:59.844 INFO:journalctl@ceph.mon.b.smithi149.stdout: s = self.context.wrap_socket( 2024-09-10T19:08:59.844 INFO:journalctl@ceph.mon.b.smithi149.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T19:08:59.844 INFO:journalctl@ceph.mon.b.smithi149.stdout: return self.sslsocket_class._create( 2024-09-10T19:08:59.844 INFO:journalctl@ceph.mon.b.smithi149.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T19:08:59.844 INFO:journalctl@ceph.mon.b.smithi149.stdout: self.do_handshake() 2024-09-10T19:08:59.844 INFO:journalctl@ceph.mon.b.smithi149.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T19:08:59.844 INFO:journalctl@ceph.mon.b.smithi149.stdout: self._sslobj.do_handshake() 2024-09-10T19:08:59.845 INFO:journalctl@ceph.mon.b.smithi149.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T19:08:59.845 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:08:59 smithi149 ceph-mon[33230]: from='mgr.24448 ' entity='mgr.x' 2024-09-10T19:09:00.618 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:00 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:09:00.358+0000 7f8de86abec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T19:09:00.618 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:00 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:09:00.475+0000 7f8de86abec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T19:09:00.710 INFO:teuthology.orchestra.run.smithi047.stderr:++ ceph auth get-key mgr.y 2024-09-10T19:09:00.843 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:00 smithi149 ceph-mon[33230]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:09:00.843 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:00 smithi149 ceph-mon[33230]: mgrmap e25: x(active, since 2s) 2024-09-10T19:09:00.875 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:00 smithi047 ceph-mon[29019]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:09:00.875 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:00 smithi047 ceph-mon[29019]: mgrmap e25: x(active, since 2s) 2024-09-10T19:09:00.875 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:00 smithi047 ceph-mon[39856]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:09:00.875 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:00 smithi047 ceph-mon[39856]: mgrmap e25: x(active, since 2s) 2024-09-10T19:09:00.875 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:00 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:09:00.616+0000 7f8de86abec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T19:09:00.876 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:00 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:09:00.680+0000 7f8de86abec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T19:09:00.876 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:00 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:09:00.817+0000 7f8de86abec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T19:09:01.035 INFO:teuthology.orchestra.run.smithi047.stdout:rotating key for mgr.x 2024-09-10T19:09:01.036 INFO:teuthology.orchestra.run.smithi047.stderr:+ NK=AQBHmeBmrC6gLhAAI6PatEN/+/uNdRSFJ0NHBA== 2024-09-10T19:09:01.036 INFO:teuthology.orchestra.run.smithi047.stderr:+ '[' AQAsluBm0dTEKxAA8EcraitHg2mj9uyD84vn1g== == AQBHmeBmrC6gLhAAI6PatEN/+/uNdRSFJ0NHBA== ']' 2024-09-10T19:09:01.036 INFO:teuthology.orchestra.run.smithi047.stderr:+ for f in osd.0 osd.1 osd.2 osd.3 osd.4 osd.5 osd.6 osd.7 mgr.y mgr.x 2024-09-10T19:09:01.036 INFO:teuthology.orchestra.run.smithi047.stderr:+ echo 'rotating key for mgr.x' 2024-09-10T19:09:01.036 INFO:teuthology.orchestra.run.smithi047.stderr:++ ceph auth get-key mgr.x 2024-09-10T19:09:01.160 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:00 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:09:00.891+0000 7f8de86abec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T19:09:01.160 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:00 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:09:00.952+0000 7f8de86abec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T19:09:01.160 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:01 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:09:01.096+0000 7f8de86abec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T19:09:01.378 INFO:teuthology.orchestra.run.smithi047.stderr:+ K=AQCeluBmO+P/LhAAaSNb0dYJwXCCva+uQ1RUmA== 2024-09-10T19:09:01.379 INFO:teuthology.orchestra.run.smithi047.stderr:+ NK=AQCeluBmO+P/LhAAaSNb0dYJwXCCva+uQ1RUmA== 2024-09-10T19:09:01.379 INFO:teuthology.orchestra.run.smithi047.stderr:+ ceph orch daemon rotate-key mgr.x 2024-09-10T19:09:01.475 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:01 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:09:01.182+0000 7f8de86abec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T19:09:01.476 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:01 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:09:01.251+0000 7f8de86abec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T19:09:01.476 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:01 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:09:01.335+0000 7f8de86abec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T19:09:01.476 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:01 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:09:01.401+0000 7f8de86abec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T19:09:01.476 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:01 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:09:01.473+0000 7f8de86abec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T19:09:01.662 INFO:teuthology.orchestra.run.smithi047.stdout:Scheduled to rotate-key mgr.x on host 'smithi149' 2024-09-10T19:09:01.672 INFO:teuthology.orchestra.run.smithi047.stderr:+ '[' AQCeluBmO+P/LhAAaSNb0dYJwXCCva+uQ1RUmA== == AQCeluBmO+P/LhAAaSNb0dYJwXCCva+uQ1RUmA== ']' 2024-09-10T19:09:01.672 INFO:teuthology.orchestra.run.smithi047.stderr:+ sleep 5 2024-09-10T19:09:01.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:01 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/3122401308' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.y"}]: dispatch 2024-09-10T19:09:01.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:01 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/3342469167' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.x"}]: dispatch 2024-09-10T19:09:01.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:01 smithi047 ceph-mon[29019]: from='mgr.24448 ' entity='mgr.x' 2024-09-10T19:09:01.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:01 smithi047 ceph-mon[29019]: from='mgr.24448 ' entity='mgr.x' 2024-09-10T19:09:01.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:01 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/3122401308' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.y"}]: dispatch 2024-09-10T19:09:01.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:01 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/3342469167' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.x"}]: dispatch 2024-09-10T19:09:01.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:01 smithi047 ceph-mon[39856]: from='mgr.24448 ' entity='mgr.x' 2024-09-10T19:09:01.790 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:01 smithi047 ceph-mon[39856]: from='mgr.24448 ' entity='mgr.x' 2024-09-10T19:09:02.006 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:01 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/3122401308' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.y"}]: dispatch 2024-09-10T19:09:02.006 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:01 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/3342469167' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.x"}]: dispatch 2024-09-10T19:09:02.006 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:01 smithi149 ceph-mon[33230]: from='mgr.24448 ' entity='mgr.x' 2024-09-10T19:09:02.006 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:01 smithi149 ceph-mon[33230]: from='mgr.24448 ' entity='mgr.x' 2024-09-10T19:09:02.628 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:02 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:09:02.299+0000 7f8de86abec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T19:09:02.628 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:02 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: [10/Sep/2024:19:09:02] ENGINE Bus STARTING 2024-09-10T19:09:02.628 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:02 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: CherryPy Checker: 2024-09-10T19:09:02.629 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:02 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: The Application mounted at '' has an empty config. 2024-09-10T19:09:02.629 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:02 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: 2024-09-10T19:09:02.629 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:02 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: [10/Sep/2024:19:09:02] ENGINE Serving on http://:::9283 2024-09-10T19:09:02.629 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:02 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: [10/Sep/2024:19:09:02] ENGINE Bus STARTED 2024-09-10T19:09:02.905 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:02 smithi149 ceph-mon[33230]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:09:02.905 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:02 smithi149 ceph-mon[33230]: mgrmap e26: x(active, since 4s) 2024-09-10T19:09:02.905 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:02 smithi149 ceph-mon[33230]: from='client.15414 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "mgr.x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:09:02.905 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:02 smithi149 ceph-mon[33230]: Schedule rotate-key daemon mgr.x 2024-09-10T19:09:02.905 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:02 smithi149 ceph-mon[33230]: from='mgr.24448 ' entity='mgr.x' 2024-09-10T19:09:02.905 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:02 smithi149 ceph-mon[33230]: from='mgr.24448 ' entity='mgr.x' 2024-09-10T19:09:02.905 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:02 smithi149 ceph-mon[33230]: from='mgr.24448 ' entity='mgr.x' 2024-09-10T19:09:02.905 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:02 smithi149 ceph-mon[33230]: from='mgr.24448 ' entity='mgr.x' 2024-09-10T19:09:02.905 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:02 smithi149 ceph-mon[33230]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:09:02.906 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:02 smithi149 ceph-mon[33230]: from='mgr.24448 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:09:02.906 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:02 smithi149 ceph-mon[33230]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:09:02.906 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:02 smithi149 ceph-mon[33230]: from='mgr.24448 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:09:02.906 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:02 smithi149 ceph-mon[33230]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:09:02.906 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:02 smithi149 ceph-mon[33230]: from='mgr.24448 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:09:02.906 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:02 smithi149 ceph-mon[33230]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:09:02.906 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:02 smithi149 ceph-mon[33230]: from='mgr.24448 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:09:02.906 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:02 smithi149 ceph-mon[33230]: Adjusting osd_memory_target on smithi047 to 3249M 2024-09-10T19:09:02.906 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:02 smithi149 ceph-mon[33230]: from='mgr.24448 ' entity='mgr.x' 2024-09-10T19:09:02.906 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:02 smithi149 ceph-mon[33230]: from='mgr.24448 ' entity='mgr.x' 2024-09-10T19:09:02.906 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:02 smithi149 ceph-mon[33230]: from='mgr.24448 ' entity='mgr.x' 2024-09-10T19:09:02.906 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:02 smithi149 ceph-mon[33230]: Standby manager daemon y started 2024-09-10T19:09:02.906 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:02 smithi149 ceph-mon[33230]: from='mgr.? 172.21.15.47:0/818745650' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-10T19:09:02.907 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:02 smithi149 ceph-mon[33230]: from='mgr.? 172.21.15.47:0/818745650' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T19:09:02.907 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:02 smithi149 ceph-mon[33230]: from='mgr.? 172.21.15.47:0/818745650' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-10T19:09:02.907 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:02 smithi149 ceph-mon[33230]: from='mgr.? 172.21.15.47:0/818745650' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T19:09:03.039 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:02 smithi047 ceph-mon[39856]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:09:03.040 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:02 smithi047 ceph-mon[39856]: mgrmap e26: x(active, since 4s) 2024-09-10T19:09:03.040 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:02 smithi047 ceph-mon[39856]: from='client.15414 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "mgr.x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:09:03.040 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:02 smithi047 ceph-mon[39856]: Schedule rotate-key daemon mgr.x 2024-09-10T19:09:03.040 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:02 smithi047 ceph-mon[39856]: from='mgr.24448 ' entity='mgr.x' 2024-09-10T19:09:03.040 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:02 smithi047 ceph-mon[39856]: from='mgr.24448 ' entity='mgr.x' 2024-09-10T19:09:03.040 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:02 smithi047 ceph-mon[39856]: from='mgr.24448 ' entity='mgr.x' 2024-09-10T19:09:03.040 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:02 smithi047 ceph-mon[39856]: from='mgr.24448 ' entity='mgr.x' 2024-09-10T19:09:03.040 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:02 smithi047 ceph-mon[39856]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:09:03.040 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:02 smithi047 ceph-mon[39856]: from='mgr.24448 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:09:03.040 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:02 smithi047 ceph-mon[39856]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:09:03.040 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:02 smithi047 ceph-mon[39856]: from='mgr.24448 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:09:03.041 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:02 smithi047 ceph-mon[39856]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:09:03.041 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:02 smithi047 ceph-mon[39856]: from='mgr.24448 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:09:03.041 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:02 smithi047 ceph-mon[39856]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:09:03.041 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:02 smithi047 ceph-mon[39856]: from='mgr.24448 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:09:03.041 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:02 smithi047 ceph-mon[39856]: Adjusting osd_memory_target on smithi047 to 3249M 2024-09-10T19:09:03.041 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:02 smithi047 ceph-mon[39856]: from='mgr.24448 ' entity='mgr.x' 2024-09-10T19:09:03.041 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:02 smithi047 ceph-mon[39856]: from='mgr.24448 ' entity='mgr.x' 2024-09-10T19:09:03.041 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:02 smithi047 ceph-mon[39856]: from='mgr.24448 ' entity='mgr.x' 2024-09-10T19:09:03.041 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:02 smithi047 ceph-mon[39856]: Standby manager daemon y started 2024-09-10T19:09:03.041 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:02 smithi047 ceph-mon[39856]: from='mgr.? 172.21.15.47:0/818745650' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-10T19:09:03.041 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:02 smithi047 ceph-mon[39856]: from='mgr.? 172.21.15.47:0/818745650' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T19:09:03.041 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:02 smithi047 ceph-mon[39856]: from='mgr.? 172.21.15.47:0/818745650' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-10T19:09:03.041 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:02 smithi047 ceph-mon[39856]: from='mgr.? 172.21.15.47:0/818745650' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T19:09:03.042 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:02 smithi047 ceph-mon[29019]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:09:03.042 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:02 smithi047 ceph-mon[29019]: mgrmap e26: x(active, since 4s) 2024-09-10T19:09:03.042 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:02 smithi047 ceph-mon[29019]: from='client.15414 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "mgr.x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:09:03.042 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:02 smithi047 ceph-mon[29019]: Schedule rotate-key daemon mgr.x 2024-09-10T19:09:03.042 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:02 smithi047 ceph-mon[29019]: from='mgr.24448 ' entity='mgr.x' 2024-09-10T19:09:03.042 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:02 smithi047 ceph-mon[29019]: from='mgr.24448 ' entity='mgr.x' 2024-09-10T19:09:03.042 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:02 smithi047 ceph-mon[29019]: from='mgr.24448 ' entity='mgr.x' 2024-09-10T19:09:03.042 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:02 smithi047 ceph-mon[29019]: from='mgr.24448 ' entity='mgr.x' 2024-09-10T19:09:03.042 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:02 smithi047 ceph-mon[29019]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:09:03.042 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:02 smithi047 ceph-mon[29019]: from='mgr.24448 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:09:03.042 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:02 smithi047 ceph-mon[29019]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:09:03.043 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:02 smithi047 ceph-mon[29019]: from='mgr.24448 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:09:03.043 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:02 smithi047 ceph-mon[29019]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:09:03.043 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:02 smithi047 ceph-mon[29019]: from='mgr.24448 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:09:03.043 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:02 smithi047 ceph-mon[29019]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:09:03.043 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:02 smithi047 ceph-mon[29019]: from='mgr.24448 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:09:03.043 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:02 smithi047 ceph-mon[29019]: Adjusting osd_memory_target on smithi047 to 3249M 2024-09-10T19:09:03.043 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:02 smithi047 ceph-mon[29019]: from='mgr.24448 ' entity='mgr.x' 2024-09-10T19:09:03.043 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:02 smithi047 ceph-mon[29019]: from='mgr.24448 ' entity='mgr.x' 2024-09-10T19:09:03.043 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:02 smithi047 ceph-mon[29019]: from='mgr.24448 ' entity='mgr.x' 2024-09-10T19:09:03.043 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:02 smithi047 ceph-mon[29019]: Standby manager daemon y started 2024-09-10T19:09:03.043 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:02 smithi047 ceph-mon[29019]: from='mgr.? 172.21.15.47:0/818745650' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-10T19:09:03.043 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:02 smithi047 ceph-mon[29019]: from='mgr.? 172.21.15.47:0/818745650' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T19:09:03.043 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:02 smithi047 ceph-mon[29019]: from='mgr.? 172.21.15.47:0/818745650' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-10T19:09:03.044 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:02 smithi047 ceph-mon[29019]: from='mgr.? 172.21.15.47:0/818745650' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T19:09:03.342 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:09:02 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: ::ffff:172.21.15.149 - - [10/Sep/2024:19:09:02] "GET /metrics HTTP/1.1" 200 34919 "" "Prometheus/2.43.0" 2024-09-10T19:09:04.242 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:03 smithi047 ceph-mon[39856]: from='mgr.24448 ' entity='mgr.x' 2024-09-10T19:09:04.242 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:03 smithi047 ceph-mon[39856]: from='mgr.24448 ' entity='mgr.x' 2024-09-10T19:09:04.243 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:03 smithi047 ceph-mon[39856]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:09:04.243 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:03 smithi047 ceph-mon[39856]: from='mgr.24448 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:09:04.243 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:03 smithi047 ceph-mon[39856]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:09:04.243 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:03 smithi047 ceph-mon[39856]: from='mgr.24448 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:09:04.243 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:03 smithi047 ceph-mon[39856]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:09:04.243 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:03 smithi047 ceph-mon[39856]: from='mgr.24448 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:09:04.243 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:03 smithi047 ceph-mon[39856]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:09:04.243 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:03 smithi047 ceph-mon[39856]: from='mgr.24448 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:09:04.243 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:03 smithi047 ceph-mon[39856]: Adjusting osd_memory_target on smithi149 to 429.8M 2024-09-10T19:09:04.243 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:03 smithi047 ceph-mon[39856]: Unable to set osd_memory_target on smithi149 to 450711756: error parsing value: Value '450711756' is below minimum 939524096 2024-09-10T19:09:04.243 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:03 smithi047 ceph-mon[39856]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:09:04.243 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:03 smithi047 ceph-mon[39856]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:09:04.243 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:03 smithi047 ceph-mon[39856]: Updating smithi047:/etc/ceph/ceph.conf 2024-09-10T19:09:04.244 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:03 smithi047 ceph-mon[39856]: Updating smithi149:/etc/ceph/ceph.conf 2024-09-10T19:09:04.244 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:03 smithi047 ceph-mon[39856]: Updating smithi149:/var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T19:09:04.244 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:03 smithi047 ceph-mon[39856]: Updating smithi047:/var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T19:09:04.244 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:03 smithi047 ceph-mon[39856]: mgrmap e27: x(active, since 5s), standbys: y 2024-09-10T19:09:04.244 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:03 smithi047 ceph-mon[39856]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T19:09:04.244 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:03 smithi047 ceph-mon[39856]: Updating smithi149:/etc/ceph/ceph.client.admin.keyring 2024-09-10T19:09:04.244 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:03 smithi047 ceph-mon[39856]: Updating smithi047:/etc/ceph/ceph.client.admin.keyring 2024-09-10T19:09:04.244 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:03 smithi047 ceph-mon[39856]: from='mgr.24448 ' entity='mgr.x' 2024-09-10T19:09:04.244 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:03 smithi047 ceph-mon[39856]: from='mgr.24448 ' entity='mgr.x' 2024-09-10T19:09:04.244 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:03 smithi047 ceph-mon[39856]: from='mgr.24448 ' entity='mgr.x' 2024-09-10T19:09:04.244 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:03 smithi047 ceph-mon[39856]: from='mgr.24448 ' entity='mgr.x' 2024-09-10T19:09:04.244 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:03 smithi047 ceph-mon[39856]: from='mgr.24448 ' entity='mgr.x' 2024-09-10T19:09:04.244 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:03 smithi047 ceph-mon[39856]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]: dispatch 2024-09-10T19:09:04.245 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:03 smithi047 ceph-mon[39856]: from='mgr.24448 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]: dispatch 2024-09-10T19:09:04.245 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:03 smithi047 ceph-mon[39856]: from='mgr.24448 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]': finished 2024-09-10T19:09:04.245 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:03 smithi047 ceph-mon[29019]: from='mgr.24448 ' entity='mgr.x' 2024-09-10T19:09:04.245 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:03 smithi047 ceph-mon[29019]: from='mgr.24448 ' entity='mgr.x' 2024-09-10T19:09:04.245 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:03 smithi047 ceph-mon[29019]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:09:04.245 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:03 smithi047 ceph-mon[29019]: from='mgr.24448 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:09:04.245 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:03 smithi047 ceph-mon[29019]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:09:04.245 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:03 smithi047 ceph-mon[29019]: from='mgr.24448 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:09:04.245 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:03 smithi047 ceph-mon[29019]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:09:04.245 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:03 smithi047 ceph-mon[29019]: from='mgr.24448 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:09:04.246 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:03 smithi047 ceph-mon[29019]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:09:04.246 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:03 smithi047 ceph-mon[29019]: from='mgr.24448 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:09:04.246 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:03 smithi047 ceph-mon[29019]: Adjusting osd_memory_target on smithi149 to 429.8M 2024-09-10T19:09:04.246 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:03 smithi047 ceph-mon[29019]: Unable to set osd_memory_target on smithi149 to 450711756: error parsing value: Value '450711756' is below minimum 939524096 2024-09-10T19:09:04.246 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:03 smithi047 ceph-mon[29019]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:09:04.246 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:03 smithi047 ceph-mon[29019]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:09:04.246 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:03 smithi047 ceph-mon[29019]: Updating smithi047:/etc/ceph/ceph.conf 2024-09-10T19:09:04.246 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:03 smithi047 ceph-mon[29019]: Updating smithi149:/etc/ceph/ceph.conf 2024-09-10T19:09:04.246 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:03 smithi047 ceph-mon[29019]: Updating smithi149:/var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T19:09:04.246 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:03 smithi047 ceph-mon[29019]: Updating smithi047:/var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T19:09:04.246 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:03 smithi047 ceph-mon[29019]: mgrmap e27: x(active, since 5s), standbys: y 2024-09-10T19:09:04.247 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:03 smithi047 ceph-mon[29019]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T19:09:04.247 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:03 smithi047 ceph-mon[29019]: Updating smithi149:/etc/ceph/ceph.client.admin.keyring 2024-09-10T19:09:04.247 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:03 smithi047 ceph-mon[29019]: Updating smithi047:/etc/ceph/ceph.client.admin.keyring 2024-09-10T19:09:04.247 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:03 smithi047 ceph-mon[29019]: from='mgr.24448 ' entity='mgr.x' 2024-09-10T19:09:04.247 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:03 smithi047 ceph-mon[29019]: from='mgr.24448 ' entity='mgr.x' 2024-09-10T19:09:04.247 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:03 smithi047 ceph-mon[29019]: from='mgr.24448 ' entity='mgr.x' 2024-09-10T19:09:04.247 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:03 smithi047 ceph-mon[29019]: from='mgr.24448 ' entity='mgr.x' 2024-09-10T19:09:04.247 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:03 smithi047 ceph-mon[29019]: from='mgr.24448 ' entity='mgr.x' 2024-09-10T19:09:04.247 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:03 smithi047 ceph-mon[29019]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]: dispatch 2024-09-10T19:09:04.247 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:03 smithi047 ceph-mon[29019]: from='mgr.24448 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]: dispatch 2024-09-10T19:09:04.247 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:03 smithi047 ceph-mon[29019]: from='mgr.24448 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]': finished 2024-09-10T19:09:04.248 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:03 smithi047 ceph-mon[39856]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T19:09:04.248 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:03 smithi047 ceph-mon[39856]: from='mgr.24448 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T19:09:04.248 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:03 smithi047 ceph-mon[39856]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T19:09:04.248 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:03 smithi047 ceph-mon[39856]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:09:04.248 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:03 smithi047 ceph-mon[29019]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T19:09:04.248 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:03 smithi047 ceph-mon[29019]: from='mgr.24448 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T19:09:04.248 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:03 smithi047 ceph-mon[29019]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T19:09:04.248 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:03 smithi047 ceph-mon[29019]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:09:04.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:03 smithi149 ceph-mon[33230]: from='mgr.24448 ' entity='mgr.x' 2024-09-10T19:09:04.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:03 smithi149 ceph-mon[33230]: from='mgr.24448 ' entity='mgr.x' 2024-09-10T19:09:04.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:03 smithi149 ceph-mon[33230]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:09:04.344 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:03 smithi149 ceph-mon[33230]: from='mgr.24448 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:09:04.344 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:03 smithi149 ceph-mon[33230]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:09:04.344 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:03 smithi149 ceph-mon[33230]: from='mgr.24448 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:09:04.344 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:03 smithi149 ceph-mon[33230]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:09:04.344 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:03 smithi149 ceph-mon[33230]: from='mgr.24448 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:09:04.344 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:03 smithi149 ceph-mon[33230]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:09:04.344 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:03 smithi149 ceph-mon[33230]: from='mgr.24448 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:09:04.345 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:03 smithi149 ceph-mon[33230]: Adjusting osd_memory_target on smithi149 to 429.8M 2024-09-10T19:09:04.345 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:03 smithi149 ceph-mon[33230]: Unable to set osd_memory_target on smithi149 to 450711756: error parsing value: Value '450711756' is below minimum 939524096 2024-09-10T19:09:04.345 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:03 smithi149 ceph-mon[33230]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:09:04.345 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:03 smithi149 ceph-mon[33230]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:09:04.345 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:03 smithi149 ceph-mon[33230]: Updating smithi047:/etc/ceph/ceph.conf 2024-09-10T19:09:04.345 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:03 smithi149 ceph-mon[33230]: Updating smithi149:/etc/ceph/ceph.conf 2024-09-10T19:09:04.345 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:03 smithi149 ceph-mon[33230]: Updating smithi149:/var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T19:09:04.345 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:03 smithi149 ceph-mon[33230]: Updating smithi047:/var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T19:09:04.345 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:03 smithi149 ceph-mon[33230]: mgrmap e27: x(active, since 5s), standbys: y 2024-09-10T19:09:04.346 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:03 smithi149 ceph-mon[33230]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T19:09:04.346 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:03 smithi149 ceph-mon[33230]: Updating smithi149:/etc/ceph/ceph.client.admin.keyring 2024-09-10T19:09:04.346 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:03 smithi149 ceph-mon[33230]: Updating smithi047:/etc/ceph/ceph.client.admin.keyring 2024-09-10T19:09:04.346 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:03 smithi149 ceph-mon[33230]: from='mgr.24448 ' entity='mgr.x' 2024-09-10T19:09:04.346 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:03 smithi149 ceph-mon[33230]: from='mgr.24448 ' entity='mgr.x' 2024-09-10T19:09:04.346 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:03 smithi149 ceph-mon[33230]: from='mgr.24448 ' entity='mgr.x' 2024-09-10T19:09:04.346 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:03 smithi149 ceph-mon[33230]: from='mgr.24448 ' entity='mgr.x' 2024-09-10T19:09:04.346 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:03 smithi149 ceph-mon[33230]: from='mgr.24448 ' entity='mgr.x' 2024-09-10T19:09:04.346 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:03 smithi149 ceph-mon[33230]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]: dispatch 2024-09-10T19:09:04.346 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:03 smithi149 ceph-mon[33230]: from='mgr.24448 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]: dispatch 2024-09-10T19:09:04.346 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:03 smithi149 ceph-mon[33230]: from='mgr.24448 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]': finished 2024-09-10T19:09:04.346 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:03 smithi149 ceph-mon[33230]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T19:09:04.346 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:03 smithi149 ceph-mon[33230]: from='mgr.24448 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T19:09:04.346 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:03 smithi149 ceph-mon[33230]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T19:09:04.347 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:03 smithi149 ceph-mon[33230]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:09:05.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:05 smithi047 ceph-mon[29019]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:09:05.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:05 smithi047 ceph-mon[29019]: Updating smithi149:/var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T19:09:05.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:05 smithi047 ceph-mon[29019]: Updating smithi047:/var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T19:09:05.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:05 smithi047 ceph-mon[29019]: Rotating authentication key for mgr.y 2024-09-10T19:09:05.540 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:05 smithi047 ceph-mon[29019]: Reconfiguring daemon mgr.y on smithi047 2024-09-10T19:09:05.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:05 smithi047 ceph-mon[39856]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:09:05.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:05 smithi047 ceph-mon[39856]: Updating smithi149:/var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T19:09:05.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:05 smithi047 ceph-mon[39856]: Updating smithi047:/var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T19:09:05.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:05 smithi047 ceph-mon[39856]: Rotating authentication key for mgr.y 2024-09-10T19:09:05.540 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:05 smithi047 ceph-mon[39856]: Reconfiguring daemon mgr.y on smithi047 2024-09-10T19:09:05.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:05 smithi149 ceph-mon[33230]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:09:05.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:05 smithi149 ceph-mon[33230]: Updating smithi149:/var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T19:09:05.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:05 smithi149 ceph-mon[33230]: Updating smithi047:/var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T19:09:05.594 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:05 smithi149 ceph-mon[33230]: Rotating authentication key for mgr.y 2024-09-10T19:09:05.594 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:05 smithi149 ceph-mon[33230]: Reconfiguring daemon mgr.y on smithi047 2024-09-10T19:09:06.039 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:05 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[29244]: ::ffff:172.21.15.149 - - [10/Sep/2024:19:09:05] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:09:06.040 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:05 smithi047 systemd[1]: Stopping Ceph mgr.y for d8563d7e-6fa5-11ef-bcea-c7b262605968... 2024-09-10T19:09:06.416 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:06 smithi047 ceph-mon[29019]: from='mgr.24448 ' entity='mgr.x' 2024-09-10T19:09:06.416 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:06 smithi047 ceph-mon[29019]: from='mgr.24448 ' entity='mgr.x' 2024-09-10T19:09:06.416 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:06 smithi047 ceph-mon[39856]: from='mgr.24448 ' entity='mgr.x' 2024-09-10T19:09:06.416 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:06 smithi047 ceph-mon[39856]: from='mgr.24448 ' entity='mgr.x' 2024-09-10T19:09:06.417 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:06 smithi047 podman[80949]: 2024-09-10 19:09:06.227398082 +0000 UTC m=+0.146804882 container died 42514f6762b3daf494541f2c64d9ad00d0adaa1657ca005d65ae41d413c6d07d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-10T19:09:06.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:06 smithi149 ceph-mon[33230]: from='mgr.24448 ' entity='mgr.x' 2024-09-10T19:09:06.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:06 smithi149 ceph-mon[33230]: from='mgr.24448 ' entity='mgr.x' 2024-09-10T19:09:06.675 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:06 smithi047 podman[80949]: 2024-09-10 19:09:06.604115721 +0000 UTC m=+0.523522531 container cleanup 42514f6762b3daf494541f2c64d9ad00d0adaa1657ca005d65ae41d413c6d07d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2) 2024-09-10T19:09:06.675 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:06 smithi047 bash[80949]: ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y 2024-09-10T19:09:06.676 INFO:teuthology.orchestra.run.smithi047.stderr:++ ceph auth get-key mgr.x 2024-09-10T19:09:07.023 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:06 smithi047 podman[80961]: 2024-09-10 19:09:06.855688939 +0000 UTC m=+0.631036567 container remove 42514f6762b3daf494541f2c64d9ad00d0adaa1657ca005d65ae41d413c6d07d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T19:09:07.024 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:06 smithi047 systemd[1]: ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-09-10T19:09:07.045 INFO:teuthology.orchestra.run.smithi047.stderr:+ NK=AQCeluBmO+P/LhAAaSNb0dYJwXCCva+uQ1RUmA== 2024-09-10T19:09:07.045 INFO:teuthology.orchestra.run.smithi047.stderr:+ '[' AQCeluBmO+P/LhAAaSNb0dYJwXCCva+uQ1RUmA== == AQCeluBmO+P/LhAAaSNb0dYJwXCCva+uQ1RUmA== ']' 2024-09-10T19:09:07.045 INFO:teuthology.orchestra.run.smithi047.stderr:+ sleep 5 2024-09-10T19:09:07.290 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:07 smithi047 systemd[1]: ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968@mgr.y.service: Failed with result 'exit-code'. 2024-09-10T19:09:07.290 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:07 smithi047 systemd[1]: Stopped Ceph mgr.y for d8563d7e-6fa5-11ef-bcea-c7b262605968. 2024-09-10T19:09:07.290 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:07 smithi047 systemd[1]: ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968@mgr.y.service: Consumed 50.943s CPU time. 2024-09-10T19:09:07.290 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:07 smithi047 systemd[1]: Starting Ceph mgr.y for d8563d7e-6fa5-11ef-bcea-c7b262605968... 2024-09-10T19:09:07.290 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:07 smithi047 podman[81088]: 2024-09-10 19:09:07.157860101 +0000 UTC m=+0.016195918 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 2024-09-10T19:09:07.290 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:07 smithi047 podman[81088]: 2024-09-10 19:09:07.256755588 +0000 UTC m=+0.115091405 container create 5f1c4720477f5b16c9491f1b52ebc7d01a326deb2ee55327457075b5d3ff8d2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T19:09:07.568 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:07 smithi047 ceph-mon[29019]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-10T19:09:07.568 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:07 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/189016694' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.x"}]: dispatch 2024-09-10T19:09:07.568 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:07 smithi047 ceph-mon[39856]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-10T19:09:07.569 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:07 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/189016694' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.x"}]: dispatch 2024-09-10T19:09:07.569 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:07 smithi047 podman[81088]: 2024-09-10 19:09:07.467514114 +0000 UTC m=+0.325849932 container init 5f1c4720477f5b16c9491f1b52ebc7d01a326deb2ee55327457075b5d3ff8d2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T19:09:07.569 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:07 smithi047 podman[81088]: 2024-09-10 19:09:07.470606079 +0000 UTC m=+0.328941906 container start 5f1c4720477f5b16c9491f1b52ebc7d01a326deb2ee55327457075b5d3ff8d2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True) 2024-09-10T19:09:07.569 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:07 smithi047 ceph-mgr[81109]: -- 172.21.15.47:0/194200355 <== mon.2 v2:172.21.15.47:3301/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55c52e3d2340 con 0x55c52e3df000 2024-09-10T19:09:07.569 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:07 smithi047 ceph-mgr[81109]: cephx client: could not set rotating key: decode_decrypt failed. error:bad magic in decode_decrypt, 233849151939156884 != 18374858748799134293 2024-09-10T19:09:07.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:07 smithi149 ceph-mon[33230]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-10T19:09:07.593 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:07 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/189016694' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.x"}]: dispatch 2024-09-10T19:09:07.853 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:07 smithi047 bash[81088]: 5f1c4720477f5b16c9491f1b52ebc7d01a326deb2ee55327457075b5d3ff8d2a 2024-09-10T19:09:07.853 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:07 smithi047 systemd[1]: Started Ceph mgr.y for d8563d7e-6fa5-11ef-bcea-c7b262605968. 2024-09-10T19:09:07.853 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:07 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[81105]: 2024-09-10T19:09:07.591+0000 7f6284353ec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T19:09:07.853 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:07 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[81105]: 2024-09-10T19:09:07.653+0000 7f6284353ec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T19:09:07.854 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:07 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[81105]: 2024-09-10T19:09:07.721+0000 7f6284353ec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T19:09:07.854 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:07 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[81105]: 2024-09-10T19:09:07.851+0000 7f6284353ec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T19:09:08.267 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:07 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[81105]: 2024-09-10T19:09:07.993+0000 7f6284353ec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T19:09:08.539 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:08 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[81105]: 2024-09-10T19:09:08.265+0000 7f6284353ec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T19:09:08.540 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:08 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[81105]: 2024-09-10T19:09:08.329+0000 7f6284353ec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T19:09:08.540 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:08 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[81105]: 2024-09-10T19:09:08.497+0000 7f6284353ec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T19:09:08.593 INFO:journalctl@ceph.iscsi.iscsi.a.smithi149.stdout:Sep 10 19:09:08 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-iscsi-iscsi-a[60538]: debug there is no tcmu-runner data available 2024-09-10T19:09:08.992 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:08 smithi149 ceph-mon[33230]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-10T19:09:08.992 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:08 smithi149 ceph-mon[33230]: from='mgr.24448 ' entity='mgr.x' 2024-09-10T19:09:08.992 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:08 smithi149 ceph-mon[33230]: from='mgr.24448 ' entity='mgr.x' 2024-09-10T19:09:08.992 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:08 smithi149 ceph-mon[33230]: Rotating authentication key for mgr.x 2024-09-10T19:09:08.992 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:08 smithi149 ceph-mon[33230]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.x", "format": "json"}]: dispatch 2024-09-10T19:09:08.992 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:08 smithi149 ceph-mon[33230]: from='mgr.24448 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.x", "format": "json"}]: dispatch 2024-09-10T19:09:08.992 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:08 smithi149 ceph-mon[33230]: from='mgr.24448 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create-pending", "entity": "mgr.x", "format": "json"}]': finished 2024-09-10T19:09:08.992 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:08 smithi149 ceph-mon[33230]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T19:09:08.992 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:08 smithi149 ceph-mon[33230]: from='mgr.24448 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T19:09:08.992 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:08 smithi149 ceph-mon[33230]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T19:09:08.993 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:08 smithi149 ceph-mon[33230]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:09:08.993 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:08 smithi149 ceph-mon[33230]: Reconfiguring daemon mgr.x on smithi149 2024-09-10T19:09:09.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:08 smithi047 ceph-mon[29019]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-10T19:09:09.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:08 smithi047 ceph-mon[29019]: from='mgr.24448 ' entity='mgr.x' 2024-09-10T19:09:09.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:08 smithi047 ceph-mon[29019]: from='mgr.24448 ' entity='mgr.x' 2024-09-10T19:09:09.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:08 smithi047 ceph-mon[29019]: Rotating authentication key for mgr.x 2024-09-10T19:09:09.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:08 smithi047 ceph-mon[29019]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.x", "format": "json"}]: dispatch 2024-09-10T19:09:09.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:08 smithi047 ceph-mon[29019]: from='mgr.24448 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.x", "format": "json"}]: dispatch 2024-09-10T19:09:09.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:08 smithi047 ceph-mon[29019]: from='mgr.24448 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create-pending", "entity": "mgr.x", "format": "json"}]': finished 2024-09-10T19:09:09.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:08 smithi047 ceph-mon[29019]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T19:09:09.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:08 smithi047 ceph-mon[29019]: from='mgr.24448 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T19:09:09.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:08 smithi047 ceph-mon[29019]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T19:09:09.040 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:08 smithi047 ceph-mon[29019]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:09:09.041 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:08 smithi047 ceph-mon[29019]: Reconfiguring daemon mgr.x on smithi149 2024-09-10T19:09:09.041 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:08 smithi047 ceph-mon[39856]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-10T19:09:09.041 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:08 smithi047 ceph-mon[39856]: from='mgr.24448 ' entity='mgr.x' 2024-09-10T19:09:09.041 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:08 smithi047 ceph-mon[39856]: from='mgr.24448 ' entity='mgr.x' 2024-09-10T19:09:09.041 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:08 smithi047 ceph-mon[39856]: Rotating authentication key for mgr.x 2024-09-10T19:09:09.041 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:08 smithi047 ceph-mon[39856]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.x", "format": "json"}]: dispatch 2024-09-10T19:09:09.041 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:08 smithi047 ceph-mon[39856]: from='mgr.24448 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.x", "format": "json"}]: dispatch 2024-09-10T19:09:09.041 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:08 smithi047 ceph-mon[39856]: from='mgr.24448 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create-pending", "entity": "mgr.x", "format": "json"}]': finished 2024-09-10T19:09:09.041 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:08 smithi047 ceph-mon[39856]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T19:09:09.041 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:08 smithi047 ceph-mon[39856]: from='mgr.24448 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T19:09:09.041 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:08 smithi047 ceph-mon[39856]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T19:09:09.042 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:08 smithi047 ceph-mon[39856]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:09:09.042 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:08 smithi047 ceph-mon[39856]: Reconfiguring daemon mgr.x on smithi149 2024-09-10T19:09:09.308 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:09:09 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:09:09.155+0000 7f00ba5b7640 -1 mgr.server reply reply (13) Permission denied access denied: does your client key have mgr caps? See http://docs.ceph.com/en/latest/mgr/administrator/#client-authentication 2024-09-10T19:09:09.308 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:09:09 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:09:09.157+0000 7f00b7db2640 -1 log_channel(cephadm) log [ERR] : Non-zero return from ['ceph', '-k', '/var/lib/ceph/mgr/ceph-x/keyring', '-n', 'mgr.x', 'tell', 'mgr.x', 'rotate-key', '-i', '-']: 2024-09-10T19:09:09.151+0000 7f4004d11640 1 -- 172.21.15.149:0/800772058 <== mon.1 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x7f3fe8003710 con 0x7f4000104710 2024-09-10T19:09:09.309 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:09:09 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:09:09.151+0000 7f4004d11640 0 cephx client: could not set rotating key: decode_decrypt failed. error:bad magic in decode_decrypt, 6886467698215752780 != 18374858748799134293 2024-09-10T19:09:09.309 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:09:09 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:09:09.151+0000 7f4007f77640 1 -- 172.21.15.149:0/800772058 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4000104710 msgr2=0x7f4000106af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T19:09:09.309 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:09:09 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:09:09.151+0000 7f4007f77640 1 --2- 172.21.15.149:0/800772058 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4000104710 0x7f4000106af0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3ff0009cd0 tx=0x7f3ff0004d10 comp rx=0 tx=0).stop 2024-09-10T19:09:09.309 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:09:09 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:09:09.151+0000 7f4007f77640 1 -- 172.21.15.149:0/800772058 shutdown_connections 2024-09-10T19:09:09.309 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:09:09 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:09:09.151+0000 7f4007f77640 1 --2- 172.21.15.149:0/800772058 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4000109b70 0x7f400010c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:09:09.309 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:09:09 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:09:09.151+0000 7f4007f77640 1 --2- 172.21.15.149:0/800772058 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40001070c0 0x7f40001094e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:09:09.309 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:09:09 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:09:09.151+0000 7f4007f77640 1 --2- 172.21.15.149:0/800772058 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4000104710 0x7f4000106af0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:09:09.309 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:09:09 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:09:09.151+0000 7f4007f77640 1 -- 172.21.15.149:0/800772058 >> 172.21.15.149:0/800772058 conn(0x7f400006aa40 msgr2=0x7f400006ce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T19:09:09.309 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:09:09 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:09:09.151+0000 7f4007f77640 1 -- 172.21.15.149:0/800772058 shutdown_connections 2024-09-10T19:09:09.309 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:09:09 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:09:09.151+0000 7f4007f77640 1 -- 172.21.15.149:0/800772058 wait complete. 2024-09-10T19:09:09.309 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:09:09 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:09:09.152+0000 7f4007f77640 1 Processor -- start 2024-09-10T19:09:09.310 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:09:09 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:09:09.152+0000 7f4007f77640 1 -- start start 2024-09-10T19:09:09.310 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:09:09 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:09:09.152+0000 7f4007f77640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4000104710 0x7f4000117830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T19:09:09.310 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:09:09 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:09:09.152+0000 7f4007f77640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f40001070c0 0x7f4000117d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T19:09:09.310 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:09:09 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:09:09.152+0000 7f4007f77640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4000109b70 0x7f400011ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T19:09:09.310 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:09:09 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:09:09.152+0000 7f4007f77640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f400011f340 con 0x7f4000109b70 2024-09-10T19:09:09.310 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:09:09 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:09:09.152+0000 7f4007f77640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f400011f4b0 con 0x7f40001070c0 2024-09-10T19:09:09.310 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:09:09 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:09:09.152+0000 7f4007f77640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f400011f670 con 0x7f4000104710 2024-09-10T19:09:09.310 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:09:09 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:09:09.152+0000 7f4005512640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f40001070c0 0x7f4000117d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T19:09:09.310 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:09:09 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:09:09.152+0000 7f4005512640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f40001070c0 0x7f4000117d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.149:39496/0 (socket says 172.21.15.149:39496) 2024-09-10T19:09:09.310 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:09:09 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:09:09.152+0000 7f4005512640 1 -- 172.21.15.149:0/2912498726 learned_addr learned my addr 172.21.15.149:0/2912498726 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-09-10T19:09:09.310 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:09:09 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:09:09.152+0000 7f4005d13640 1 --2- 172.21.15.149:0/2912498726 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4000104710 0x7f4000117830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T19:09:09.310 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:09:09 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:09:09.152+0000 7f4006514640 1 --2- 172.21.15.149:0/2912498726 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4000109b70 0x7f400011ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T19:09:09.311 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:09:09 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:09:09.152+0000 7f4005512640 1 -- 172.21.15.149:0/2912498726 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4000104710 msgr2=0x7f4000117830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T19:09:09.311 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:09:09 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:09:09.152+0000 7f4005512640 1 --2- 172.21.15.149:0/2912498726 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4000104710 0x7f4000117830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:09:09.311 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:09:09 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:09:09.152+0000 7f4005512640 1 -- 172.21.15.149:0/2912498726 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4000109b70 msgr2=0x7f400011ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T19:09:09.311 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:09:09 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:09:09.152+0000 7f4005512640 1 --2- 172.21.15.149:0/2912498726 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4000109b70 0x7f400011ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:09:09.311 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:09:09 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:09:09.152+0000 7f4005512640 1 -- 172.21.15.149:0/2912498726 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ff00098b0 con 0x7f40001070c0 2024-09-10T19:09:09.311 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:09:09 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:09:09.152+0000 7f4006514640 1 --2- 172.21.15.149:0/2912498726 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4000109b70 0x7f400011ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T19:09:09.311 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:09:09 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:09:09.153+0000 7f4005512640 1 --2- 172.21.15.149:0/2912498726 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f40001070c0 0x7f4000117d70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3ff4002ad0 tx=0x7f3ff4004600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T19:09:09.312 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:09:09 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:09:09.153+0000 7f3ffe7fc640 1 -- 172.21.15.149:0/2912498726 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff4007c30 con 0x7f40001070c0 2024-09-10T19:09:09.312 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:09:09 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:09:09.153+0000 7f3ffe7fc640 1 -- 172.21.15.149:0/2912498726 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- auth(proto 2 2 bytes epoch 0) v1 -- 0x7f3fe8002070 con 0x7f40001070c0 2024-09-10T19:09:09.312 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:09:09 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:09:09.153+0000 7f3ffe7fc640 1 -- 172.21.15.149:0/2912498726 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(38 keys) v1 ==== 1752+0+0 (secure 0 0 0) 0x7f3ff400f040 con 0x7f40001070c0 2024-09-10T19:09:09.312 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:09:09 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:09:09.153+0000 7f4007f77640 1 -- 172.21.15.149:0/2912498726 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f400011f950 con 0x7f40001070c0 2024-09-10T19:09:09.312 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:09:09 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:09:09.153+0000 7f3ffe7fc640 1 -- 172.21.15.149:0/2912498726 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff4013870 con 0x7f40001070c0 2024-09-10T19:09:09.312 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:09:09 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:09:09.153+0000 7f4007f77640 1 -- 172.21.15.149:0/2912498726 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f400011fe20 con 0x7f40001070c0 2024-09-10T19:09:09.312 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:09:09 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:09:09.153+0000 7f3ffe7fc640 1 -- 172.21.15.149:0/2912498726 <== mon.1 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x7f400011fe20 con 0x7f40001070c0 2024-09-10T19:09:09.312 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:09:09 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:09:09.153+0000 7f3ffe7fc640 0 cephx client: could not set rotating key: decode_decrypt failed. error:bad magic in decode_decrypt, 6886467698215752780 != 18374858748799134293 2024-09-10T19:09:09.312 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:09:09 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:09:09.154+0000 7f3ffe7fc640 1 -- 172.21.15.149:0/2912498726 <== mon.1 v2:172.21.15.149:3300/0 5 ==== mgrmap(e 27) v1 ==== 90804+0+0 (secure 0 0 0) 0x7f3ff4023020 con 0x7f40001070c0 2024-09-10T19:09:09.312 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:09:09 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:09:09.155+0000 7f3ffe7fc640 1 --2- 172.21.15.149:0/2912498726 >> v2:172.21.15.149:6816/2160729256 conn(0x7f3fe806f120 0x7f3fe80715d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T19:09:09.312 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:09:09 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:09:09.155+0000 7f3ffe7fc640 1 -- 172.21.15.149:0/2912498726 --> v2:172.21.15.149:6816/2160729256 -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3fe8071ca0 con 0x7f3fe806f120 2024-09-10T19:09:09.312 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:09:09 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:09:09.155+0000 7f3ffe7fc640 1 -- 172.21.15.149:0/2912498726 <== mon.1 v2:172.21.15.149:3300/0 6 ==== osd_map(65..65 src has 1..65) v4 ==== 6499+0+0 (secure 0 0 0) 0x7f3ff408ed70 con 0x7f40001070c0 2024-09-10T19:09:09.313 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:09:09 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:09:09.155+0000 7f4005d13640 1 --2- 172.21.15.149:0/2912498726 >> v2:172.21.15.149:6816/2160729256 conn(0x7f3fe806f120 0x7f3fe80715d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T19:09:09.313 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:09:09 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:09:09.155+0000 7f4005d13640 1 --2- 172.21.15.149:0/2912498726 >> v2:172.21.15.149:6816/2160729256 conn(0x7f3fe806f120 0x7f3fe80715d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4000118d60 tx=0x7f3ff0004d10 comp rx=0 tx=0).ready entity=mgr.24448 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T19:09:09.313 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:09:09 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:09:09.155+0000 7f3ffe7fc640 1 -- 172.21.15.149:0/2912498726 <== mgr.24448 v2:172.21.15.149:6816/2160729256 1 ==== command_reply(tid 0: -13 access denied: does your client key have mgr caps? See http://docs.ceph.com/en/latest/mgr/administrator/#client-authentication) v1 ==== 134+0+0 (secure 0 0 0) 0x7f3fe8071ca0 con 0x7f3fe806f120 2024-09-10T19:09:09.313 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:09:09 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: Error EACCES: problem getting command descriptions from mgr.x 2024-09-10T19:09:09.313 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:09:09 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:09:09.313 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:09:09 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:09:09.168+0000 7f00dd1de640 -1 mgr handle_mgr_map I was active but no longer am 2024-09-10T19:09:09.313 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:09:09 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: ignoring --setuser ceph since I am not root 2024-09-10T19:09:09.313 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:09:09 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: ignoring --setgroup ceph since I am not root 2024-09-10T19:09:09.313 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:09:09 smithi149 ceph-mgr[35094]: -- 172.21.15.149:0/2050952147 <== mon.0 v2:172.21.15.47:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d48fa88340 con 0x55d48fa95000 2024-09-10T19:09:09.313 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:09:09 smithi149 ceph-mgr[35094]: cephx client: could not set rotating key: decode_decrypt failed. error:bad magic in decode_decrypt, 6886467698215752780 != 18374858748799134293 2024-09-10T19:09:09.313 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:09:09 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:09:09.306+0000 7f532f2b0ec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T19:09:09.468 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:09 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[81105]: 2024-09-10T19:09:09.398+0000 7f6284353ec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T19:09:09.593 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:09:09 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:09:09.361+0000 7f532f2b0ec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T19:09:09.593 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:09:09 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:09:09.424+0000 7f532f2b0ec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T19:09:09.593 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:09:09 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:09:09.546+0000 7f532f2b0ec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T19:09:09.789 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:09 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[81105]: 2024-09-10T19:09:09.467+0000 7f6284353ec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T19:09:09.936 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:09:09 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:09:09.681+0000 7f532f2b0ec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T19:09:10.339 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:10 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[81105]: 2024-09-10T19:09:10.221+0000 7f6284353ec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T19:09:10.339 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:10 smithi047 ceph-mon[39856]: from='mgr.24448 ' entity='mgr.x' 2024-09-10T19:09:10.340 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:10 smithi047 ceph-mon[39856]: from='mgr.24448 ' entity='mgr.x' 2024-09-10T19:09:10.340 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:10 smithi047 ceph-mon[39856]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "mgr fail", "who": "x"}]: dispatch 2024-09-10T19:09:10.340 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:10 smithi047 ceph-mon[39856]: from='mgr.24448 ' entity='mgr.x' cmd=[{"prefix": "mgr fail", "who": "x"}]: dispatch 2024-09-10T19:09:10.340 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:10 smithi047 ceph-mon[39856]: Activating manager daemon y 2024-09-10T19:09:10.340 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:10 smithi047 ceph-mon[39856]: osdmap e66: 8 total, 8 up, 8 in 2024-09-10T19:09:10.340 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:10 smithi047 ceph-mon[39856]: from='mgr.24448 ' entity='mgr.x' cmd='[{"prefix": "mgr fail", "who": "x"}]': finished 2024-09-10T19:09:10.340 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:10 smithi047 ceph-mon[39856]: mgrmap e28: y(active, starting, since 0.00626333s) 2024-09-10T19:09:10.340 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:10 smithi047 ceph-mon[29019]: from='mgr.24448 ' entity='mgr.x' 2024-09-10T19:09:10.340 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:10 smithi047 ceph-mon[29019]: from='mgr.24448 ' entity='mgr.x' 2024-09-10T19:09:10.340 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:10 smithi047 ceph-mon[29019]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "mgr fail", "who": "x"}]: dispatch 2024-09-10T19:09:10.340 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:10 smithi047 ceph-mon[29019]: from='mgr.24448 ' entity='mgr.x' cmd=[{"prefix": "mgr fail", "who": "x"}]: dispatch 2024-09-10T19:09:10.341 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:10 smithi047 ceph-mon[29019]: Activating manager daemon y 2024-09-10T19:09:10.341 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:10 smithi047 ceph-mon[29019]: osdmap e66: 8 total, 8 up, 8 in 2024-09-10T19:09:10.341 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:10 smithi047 ceph-mon[29019]: from='mgr.24448 ' entity='mgr.x' cmd='[{"prefix": "mgr fail", "who": "x"}]': finished 2024-09-10T19:09:10.341 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:10 smithi047 ceph-mon[29019]: mgrmap e28: y(active, starting, since 0.00626333s) 2024-09-10T19:09:10.343 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:09:09 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:09:09.934+0000 7f532f2b0ec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T19:09:10.343 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:09:09 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:09:09.994+0000 7f532f2b0ec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T19:09:10.343 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:09:10 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:09:10.154+0000 7f532f2b0ec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T19:09:10.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:10 smithi149 ceph-mon[33230]: from='mgr.24448 ' entity='mgr.x' 2024-09-10T19:09:10.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:10 smithi149 ceph-mon[33230]: from='mgr.24448 ' entity='mgr.x' 2024-09-10T19:09:10.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:10 smithi149 ceph-mon[33230]: from='mgr.24448 172.21.15.149:0/1542409441' entity='mgr.x' cmd=[{"prefix": "mgr fail", "who": "x"}]: dispatch 2024-09-10T19:09:10.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:10 smithi149 ceph-mon[33230]: from='mgr.24448 ' entity='mgr.x' cmd=[{"prefix": "mgr fail", "who": "x"}]: dispatch 2024-09-10T19:09:10.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:10 smithi149 ceph-mon[33230]: Activating manager daemon y 2024-09-10T19:09:10.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:10 smithi149 ceph-mon[33230]: osdmap e66: 8 total, 8 up, 8 in 2024-09-10T19:09:10.344 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:10 smithi149 ceph-mon[33230]: from='mgr.24448 ' entity='mgr.x' cmd='[{"prefix": "mgr fail", "who": "x"}]': finished 2024-09-10T19:09:10.344 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:10 smithi149 ceph-mon[33230]: mgrmap e28: y(active, starting, since 0.00626333s) 2024-09-10T19:09:10.659 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:10 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[81105]: 2024-09-10T19:09:10.337+0000 7f6284353ec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T19:09:10.659 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:10 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[81105]: 2024-09-10T19:09:10.463+0000 7f6284353ec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T19:09:10.659 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:10 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[81105]: 2024-09-10T19:09:10.525+0000 7f6284353ec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T19:09:10.659 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:10 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[81105]: 2024-09-10T19:09:10.657+0000 7f6284353ec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T19:09:10.929 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:10 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[81105]: 2024-09-10T19:09:10.726+0000 7f6284353ec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T19:09:10.929 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:10 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[81105]: 2024-09-10T19:09:10.787+0000 7f6284353ec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T19:09:10.929 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:10 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[81105]: 2024-09-10T19:09:10.927+0000 7f6284353ec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T19:09:11.213 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:10 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[81105]: 2024-09-10T19:09:10.993+0000 7f6284353ec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T19:09:11.213 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:11 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[81105]: 2024-09-10T19:09:11.061+0000 7f6284353ec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T19:09:11.213 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:11 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[81105]: 2024-09-10T19:09:11.148+0000 7f6284353ec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T19:09:11.214 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:11 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[81105]: 2024-09-10T19:09:11.207+0000 7f6284353ec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T19:09:11.343 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:09:10 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:09:10.992+0000 7f532f2b0ec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T19:09:11.343 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:09:11 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:09:11.057+0000 7f532f2b0ec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T19:09:11.539 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:11 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[81105]: 2024-09-10T19:09:11.267+0000 7f6284353ec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T19:09:12.047 INFO:teuthology.orchestra.run.smithi047.stderr:++ ceph auth get-key mgr.x 2024-09-10T19:09:12.060 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:09:11 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:09:11.766+0000 7f532f2b0ec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T19:09:12.060 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:09:11 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:09:11.877+0000 7f532f2b0ec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T19:09:12.060 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:09:11 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:09:11.998+0000 7f532f2b0ec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T19:09:12.060 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:09:12 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:09:12.058+0000 7f532f2b0ec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T19:09:12.316 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:09:12 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:09:12.186+0000 7f532f2b0ec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T19:09:12.316 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:09:12 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:09:12.255+0000 7f532f2b0ec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T19:09:12.316 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:09:12 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:09:12.314+0000 7f532f2b0ec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T19:09:12.316 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:12 smithi149 ceph-mon[33230]: Active manager daemon y restarted 2024-09-10T19:09:12.316 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:12 smithi149 ceph-mon[33230]: Activating manager daemon y 2024-09-10T19:09:12.317 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:12 smithi149 ceph-mon[33230]: osdmap e67: 8 total, 8 up, 8 in 2024-09-10T19:09:12.317 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:12 smithi149 ceph-mon[33230]: mgrmap e29: y(active, starting, since 0.00584283s) 2024-09-10T19:09:12.317 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:12 smithi149 ceph-mon[33230]: from='mgr.25331 172.21.15.47:0/641999239' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-10T19:09:12.317 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:12 smithi149 ceph-mon[33230]: from='mgr.25331 172.21.15.47:0/641999239' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T19:09:12.317 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:12 smithi149 ceph-mon[33230]: from='mgr.25331 172.21.15.47:0/641999239' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T19:09:12.317 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:12 smithi149 ceph-mon[33230]: from='mgr.25331 172.21.15.47:0/641999239' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T19:09:12.317 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:12 smithi149 ceph-mon[33230]: from='mgr.25331 172.21.15.47:0/641999239' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T19:09:12.317 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:12 smithi149 ceph-mon[33230]: from='mgr.25331 172.21.15.47:0/641999239' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T19:09:12.317 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:12 smithi149 ceph-mon[33230]: from='mgr.25331 172.21.15.47:0/641999239' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T19:09:12.317 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:12 smithi149 ceph-mon[33230]: from='mgr.25331 172.21.15.47:0/641999239' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T19:09:12.317 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:12 smithi149 ceph-mon[33230]: from='mgr.25331 172.21.15.47:0/641999239' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T19:09:12.317 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:12 smithi149 ceph-mon[33230]: from='mgr.25331 172.21.15.47:0/641999239' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T19:09:12.317 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:12 smithi149 ceph-mon[33230]: from='mgr.25331 172.21.15.47:0/641999239' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T19:09:12.318 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:12 smithi149 ceph-mon[33230]: from='mgr.25331 172.21.15.47:0/641999239' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T19:09:12.318 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:12 smithi149 ceph-mon[33230]: from='mgr.25331 172.21.15.47:0/641999239' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T19:09:12.318 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:12 smithi149 ceph-mon[33230]: from='mgr.25331 172.21.15.47:0/641999239' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T19:09:12.318 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:12 smithi149 ceph-mon[33230]: from='mgr.25331 172.21.15.47:0/641999239' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T19:09:12.318 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:12 smithi149 ceph-mon[33230]: from='mgr.25331 172.21.15.47:0/641999239' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T19:09:12.318 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:12 smithi149 ceph-mon[33230]: from='mgr.25331 172.21.15.47:0/641999239' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T19:09:12.318 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:12 smithi149 ceph-mon[33230]: from='mgr.25331 172.21.15.47:0/641999239' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-10T19:09:12.318 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:12 smithi149 ceph-mon[33230]: from='mgr.25331 172.21.15.47:0/641999239' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T19:09:12.371 INFO:teuthology.orchestra.run.smithi047.stderr:+ NK=AQBTmeBm0ERNJBAAH2irXK13ueBfrlCeK6bGQg== 2024-09-10T19:09:12.371 INFO:teuthology.orchestra.run.smithi047.stderr:+ '[' AQCeluBmO+P/LhAAaSNb0dYJwXCCva+uQ1RUmA== == AQBTmeBm0ERNJBAAH2irXK13ueBfrlCeK6bGQg== ']' 2024-09-10T19:09:12.372 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:12 smithi047 ceph-mon[29019]: Active manager daemon y restarted 2024-09-10T19:09:12.372 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:12 smithi047 ceph-mon[29019]: Activating manager daemon y 2024-09-10T19:09:12.372 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:12 smithi047 ceph-mon[29019]: osdmap e67: 8 total, 8 up, 8 in 2024-09-10T19:09:12.372 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:12 smithi047 ceph-mon[29019]: mgrmap e29: y(active, starting, since 0.00584283s) 2024-09-10T19:09:12.372 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:12 smithi047 ceph-mon[29019]: from='mgr.25331 172.21.15.47:0/641999239' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-10T19:09:12.372 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:12 smithi047 ceph-mon[29019]: from='mgr.25331 172.21.15.47:0/641999239' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T19:09:12.372 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:12 smithi047 ceph-mon[29019]: from='mgr.25331 172.21.15.47:0/641999239' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T19:09:12.372 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:12 smithi047 ceph-mon[29019]: from='mgr.25331 172.21.15.47:0/641999239' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T19:09:12.372 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:12 smithi047 ceph-mon[29019]: from='mgr.25331 172.21.15.47:0/641999239' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T19:09:12.373 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:12 smithi047 ceph-mon[29019]: from='mgr.25331 172.21.15.47:0/641999239' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T19:09:12.373 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:12 smithi047 ceph-mon[29019]: from='mgr.25331 172.21.15.47:0/641999239' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T19:09:12.373 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:12 smithi047 ceph-mon[29019]: from='mgr.25331 172.21.15.47:0/641999239' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T19:09:12.373 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:12 smithi047 ceph-mon[29019]: from='mgr.25331 172.21.15.47:0/641999239' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T19:09:12.373 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:12 smithi047 ceph-mon[29019]: from='mgr.25331 172.21.15.47:0/641999239' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T19:09:12.373 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:12 smithi047 ceph-mon[29019]: from='mgr.25331 172.21.15.47:0/641999239' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T19:09:12.373 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:12 smithi047 ceph-mon[29019]: from='mgr.25331 172.21.15.47:0/641999239' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T19:09:12.373 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:12 smithi047 ceph-mon[29019]: from='mgr.25331 172.21.15.47:0/641999239' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T19:09:12.373 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:12 smithi047 ceph-mon[29019]: from='mgr.25331 172.21.15.47:0/641999239' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T19:09:12.373 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:12 smithi047 ceph-mon[29019]: from='mgr.25331 172.21.15.47:0/641999239' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T19:09:12.373 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:12 smithi047 ceph-mon[29019]: from='mgr.25331 172.21.15.47:0/641999239' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T19:09:12.373 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:12 smithi047 ceph-mon[29019]: from='mgr.25331 172.21.15.47:0/641999239' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T19:09:12.373 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:12 smithi047 ceph-mon[29019]: from='mgr.25331 172.21.15.47:0/641999239' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-10T19:09:12.373 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:12 smithi047 ceph-mon[29019]: from='mgr.25331 172.21.15.47:0/641999239' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T19:09:12.374 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:12 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[81105]: 2024-09-10T19:09:12.048+0000 7f6284353ec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T19:09:12.374 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:12 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[81105]: [10/Sep/2024:19:09:12] ENGINE Bus STARTING 2024-09-10T19:09:12.374 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:12 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[81105]: CherryPy Checker: 2024-09-10T19:09:12.374 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:12 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[81105]: The Application mounted at '' has an empty config. 2024-09-10T19:09:12.374 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:12 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[81105]: 2024-09-10T19:09:12.374 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:12 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[81105]: [10/Sep/2024:19:09:12] ENGINE Serving on http://:::9283 2024-09-10T19:09:12.374 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:12 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[81105]: [10/Sep/2024:19:09:12] ENGINE Bus STARTED 2024-09-10T19:09:12.374 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:12 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[81105]: [10/Sep/2024:19:09:12] ENGINE Bus STOPPING 2024-09-10T19:09:12.374 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:12 smithi047 ceph-mon[39856]: Active manager daemon y restarted 2024-09-10T19:09:12.375 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:12 smithi047 ceph-mon[39856]: Activating manager daemon y 2024-09-10T19:09:12.375 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:12 smithi047 ceph-mon[39856]: osdmap e67: 8 total, 8 up, 8 in 2024-09-10T19:09:12.375 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:12 smithi047 ceph-mon[39856]: mgrmap e29: y(active, starting, since 0.00584283s) 2024-09-10T19:09:12.375 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:12 smithi047 ceph-mon[39856]: from='mgr.25331 172.21.15.47:0/641999239' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-10T19:09:12.376 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:12 smithi047 ceph-mon[39856]: from='mgr.25331 172.21.15.47:0/641999239' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T19:09:12.376 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:12 smithi047 ceph-mon[39856]: from='mgr.25331 172.21.15.47:0/641999239' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T19:09:12.376 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:12 smithi047 ceph-mon[39856]: from='mgr.25331 172.21.15.47:0/641999239' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T19:09:12.376 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:12 smithi047 ceph-mon[39856]: from='mgr.25331 172.21.15.47:0/641999239' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T19:09:12.376 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:12 smithi047 ceph-mon[39856]: from='mgr.25331 172.21.15.47:0/641999239' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T19:09:12.376 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:12 smithi047 ceph-mon[39856]: from='mgr.25331 172.21.15.47:0/641999239' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T19:09:12.376 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:12 smithi047 ceph-mon[39856]: from='mgr.25331 172.21.15.47:0/641999239' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T19:09:12.377 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:12 smithi047 ceph-mon[39856]: from='mgr.25331 172.21.15.47:0/641999239' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T19:09:12.377 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:12 smithi047 ceph-mon[39856]: from='mgr.25331 172.21.15.47:0/641999239' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T19:09:12.377 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:12 smithi047 ceph-mon[39856]: from='mgr.25331 172.21.15.47:0/641999239' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T19:09:12.377 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:12 smithi047 ceph-mon[39856]: from='mgr.25331 172.21.15.47:0/641999239' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T19:09:12.377 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:12 smithi047 ceph-mon[39856]: from='mgr.25331 172.21.15.47:0/641999239' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T19:09:12.377 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:12 smithi047 ceph-mon[39856]: from='mgr.25331 172.21.15.47:0/641999239' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T19:09:12.377 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:12 smithi047 ceph-mon[39856]: from='mgr.25331 172.21.15.47:0/641999239' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T19:09:12.377 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:12 smithi047 ceph-mon[39856]: from='mgr.25331 172.21.15.47:0/641999239' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T19:09:12.378 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:12 smithi047 ceph-mon[39856]: from='mgr.25331 172.21.15.47:0/641999239' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T19:09:12.378 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:12 smithi047 ceph-mon[39856]: from='mgr.25331 172.21.15.47:0/641999239' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-10T19:09:12.378 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:12 smithi047 ceph-mon[39856]: from='mgr.25331 172.21.15.47:0/641999239' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T19:09:12.578 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:09:12 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:09:12.449+0000 7f532f2b0ec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T19:09:12.578 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:09:12 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:09:12.512+0000 7f532f2b0ec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T19:09:12.578 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:09:12 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:09:12.576+0000 7f532f2b0ec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T19:09:12.843 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:09:12 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:09:12.657+0000 7f532f2b0ec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T19:09:12.843 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:09:12 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:09:12.716+0000 7f532f2b0ec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T19:09:12.843 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:09:12 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:09:12.773+0000 7f532f2b0ec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T19:09:12.954 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:12 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[81105]: [10/Sep/2024:19:09:12] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-10T19:09:12.954 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:12 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[81105]: [10/Sep/2024:19:09:12] ENGINE Bus STOPPED 2024-09-10T19:09:12.954 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:12 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[81105]: [10/Sep/2024:19:09:12] ENGINE Bus STARTING 2024-09-10T19:09:12.954 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:12 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[81105]: [10/Sep/2024:19:09:12] ENGINE Serving on http://:::9283 2024-09-10T19:09:12.954 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:12 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[81105]: [10/Sep/2024:19:09:12] ENGINE Bus STARTED 2024-09-10T19:09:13.061 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-10T19:09:13.070 INFO:tasks.cephadm:Teardown begin 2024-09-10T19:09:13.071 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T19:09:13.099 DEBUG:teuthology.orchestra.run.smithi149:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T19:09:13.131 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-10T19:09:13.131 DEBUG:teuthology.orchestra.run.smithi047:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-10T19:09:13.156 DEBUG:teuthology.orchestra.run.smithi149:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-10T19:09:13.186 INFO:tasks.cephadm:Stopping all daemons... 2024-09-10T19:09:13.186 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-10T19:09:13.187 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968@mon.a 2024-09-10T19:09:13.218 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:13 smithi047 ceph-mon[39856]: from='client.? 172.21.15.47:0/1867682574' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.x"}]: dispatch 2024-09-10T19:09:13.218 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:13 smithi047 ceph-mon[39856]: Manager daemon y is now available 2024-09-10T19:09:13.218 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:13 smithi047 ceph-mon[39856]: from='mgr.25331 172.21.15.47:0/641999239' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:09:13.219 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:13 smithi047 ceph-mon[39856]: from='mgr.25331 172.21.15.47:0/641999239' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:09:13.219 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:13 smithi047 ceph-mon[39856]: from='mgr.25331 172.21.15.47:0/641999239' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-10T19:09:13.219 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:13 smithi047 ceph-mon[39856]: from='mgr.25331 172.21.15.47:0/641999239' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-10T19:09:13.219 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:13 smithi047 ceph-mon[29019]: from='client.? 172.21.15.47:0/1867682574' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.x"}]: dispatch 2024-09-10T19:09:13.219 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:13 smithi047 ceph-mon[29019]: Manager daemon y is now available 2024-09-10T19:09:13.219 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:13 smithi047 ceph-mon[29019]: from='mgr.25331 172.21.15.47:0/641999239' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:09:13.219 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:13 smithi047 ceph-mon[29019]: from='mgr.25331 172.21.15.47:0/641999239' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:09:13.219 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:13 smithi047 ceph-mon[29019]: from='mgr.25331 172.21.15.47:0/641999239' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-10T19:09:13.219 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:13 smithi047 ceph-mon[29019]: from='mgr.25331 172.21.15.47:0/641999239' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-10T19:09:13.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:13 smithi149 ceph-mon[33230]: from='client.? 172.21.15.47:0/1867682574' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.x"}]: dispatch 2024-09-10T19:09:13.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:13 smithi149 ceph-mon[33230]: Manager daemon y is now available 2024-09-10T19:09:13.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:13 smithi149 ceph-mon[33230]: from='mgr.25331 172.21.15.47:0/641999239' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:09:13.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:13 smithi149 ceph-mon[33230]: from='mgr.25331 172.21.15.47:0/641999239' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:09:13.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:13 smithi149 ceph-mon[33230]: from='mgr.25331 172.21.15.47:0/641999239' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-10T19:09:13.343 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:13 smithi149 ceph-mon[33230]: from='mgr.25331 172.21.15.47:0/641999239' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-10T19:09:13.511 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:13 smithi047 systemd[1]: Stopping Ceph mon.a for d8563d7e-6fa5-11ef-bcea-c7b262605968... 2024-09-10T19:09:13.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:13 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mon-a[29015]: 2024-09-10T19:09:13.509+0000 7f9bb9c3b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T19:09:13.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:13 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mon-a[29015]: 2024-09-10T19:09:13.509+0000 7f9bb9c3b640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-10T19:09:13.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:13 smithi047 podman[81553]: 2024-09-10 19:09:13.561155415 +0000 UTC m=+0.151461417 container died 555ddf4fbf7a1872c931d51a9e9f0afdee9a75652b73d7c2f33e750bb4b3a5e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mon-a, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-10T19:09:13.790 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:13 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[81105]: [10/Sep/2024:19:09:13] ENGINE Bus STOPPING 2024-09-10T19:09:13.843 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:09:13 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:09:13.580+0000 7f532f2b0ec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T19:09:13.843 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:09:13 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: [10/Sep/2024:19:09:13] ENGINE Bus STARTING 2024-09-10T19:09:13.843 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:09:13 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: CherryPy Checker: 2024-09-10T19:09:13.843 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:09:13 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: The Application mounted at '' has an empty config. 2024-09-10T19:09:13.843 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:09:13 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: 2024-09-10T19:09:13.843 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:09:13 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: [10/Sep/2024:19:09:13] ENGINE Serving on http://:::9283 2024-09-10T19:09:13.843 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:09:13 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x[35090]: [10/Sep/2024:19:09:13] ENGINE Bus STARTED 2024-09-10T19:09:14.176 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:13 smithi047 podman[81553]: 2024-09-10 19:09:13.950720273 +0000 UTC m=+0.541026275 container cleanup 555ddf4fbf7a1872c931d51a9e9f0afdee9a75652b73d7c2f33e750bb4b3a5e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-10T19:09:14.176 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:13 smithi047 bash[81553]: ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mon-a 2024-09-10T19:09:14.176 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:13 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[81105]: [10/Sep/2024:19:09:13] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-10T19:09:14.176 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:13 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[81105]: [10/Sep/2024:19:09:13] ENGINE Bus STOPPED 2024-09-10T19:09:14.176 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:13 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[81105]: [10/Sep/2024:19:09:13] ENGINE Bus STARTING 2024-09-10T19:09:14.176 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:13 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[81105]: [10/Sep/2024:19:09:13] ENGINE Serving on http://:::9283 2024-09-10T19:09:14.177 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:13 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[81105]: [10/Sep/2024:19:09:13] ENGINE Bus STARTED 2024-09-10T19:09:14.505 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:14 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[81105]: 2024-09-10T19:09:14.222+0000 7f6226e94640 -1 log_channel(cephadm) log [ERR] : [10/Sep/2024:19:09:14] ENGINE Error in HTTPServer.serve 2024-09-10T19:09:14.505 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:14 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[81105]: Traceback (most recent call last): 2024-09-10T19:09:14.505 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:14 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[81105]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T19:09:14.506 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:14 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[81105]: self._connections.run(self.expiration_interval) 2024-09-10T19:09:14.506 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:14 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[81105]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T19:09:14.506 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:14 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[81105]: self._run(expiration_interval) 2024-09-10T19:09:14.506 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:14 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[81105]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T19:09:14.506 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:14 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[81105]: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T19:09:14.506 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:14 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[81105]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T19:09:14.506 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:14 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[81105]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T19:09:14.506 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:14 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[81105]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T19:09:14.506 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:14 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[81105]: s = self.context.wrap_socket( 2024-09-10T19:09:14.506 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:14 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[81105]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T19:09:14.506 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:14 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[81105]: return self.sslsocket_class._create( 2024-09-10T19:09:14.506 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:14 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[81105]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T19:09:14.506 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:14 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[81105]: self.do_handshake() 2024-09-10T19:09:14.507 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:14 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[81105]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T19:09:14.507 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:14 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[81105]: self._sslobj.do_handshake() 2024-09-10T19:09:14.507 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:14 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[81105]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T19:09:14.507 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:14 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[81105]: 2024-09-10T19:09:14.789 INFO:journalctl@ceph.mon.a.smithi047.stdout:Sep 10 19:09:14 smithi047 podman[81592]: 2024-09-10 19:09:14.504193286 +0000 UTC m=+0.943626693 container remove 555ddf4fbf7a1872c931d51a9e9f0afdee9a75652b73d7c2f33e750bb4b3a5e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mon-a, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-10T19:09:14.974 DEBUG:teuthology.orchestra.run.smithi047:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968@mon.a.service' 2024-09-10T19:09:15.066 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T19:09:15.066 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-10T19:09:15.067 INFO:tasks.cephadm.mon.b:Stopping mon.c... 2024-09-10T19:09:15.067 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968@mon.c 2024-09-10T19:09:15.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:15 smithi047 systemd[1]: Stopping Ceph mon.c for d8563d7e-6fa5-11ef-bcea-c7b262605968... 2024-09-10T19:09:16.040 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:15 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[81105]: ::ffff:172.21.15.149 - - [10/Sep/2024:19:09:15] "GET /metrics HTTP/1.1" 200 34921 "" "Prometheus/2.43.0" 2024-09-10T19:09:16.309 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:16 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mon-c[39849]: 2024-09-10T19:09:16.045+0000 7fc7feafc640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T19:09:16.309 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:16 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mon-c[39849]: 2024-09-10T19:09:16.045+0000 7fc7feafc640 -1 mon.c@2(peon) e3 *** Got Signal Terminated *** 2024-09-10T19:09:16.309 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:16 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[81105]: [10/Sep/2024:19:09:16] ENGINE Bus STOPPING 2024-09-10T19:09:16.309 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:16 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[81105]: [10/Sep/2024:19:09:16] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-10T19:09:16.309 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:16 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[81105]: [10/Sep/2024:19:09:16] ENGINE Bus STOPPED 2024-09-10T19:09:16.309 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:16 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[81105]: [10/Sep/2024:19:09:16] ENGINE Bus STARTING 2024-09-10T19:09:16.309 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:16 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[81105]: [10/Sep/2024:19:09:16] ENGINE Serving on http://:::9283 2024-09-10T19:09:16.310 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:16 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y[81105]: [10/Sep/2024:19:09:16] ENGINE Bus STARTED 2024-09-10T19:09:17.009 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:16 smithi047 podman[81986]: 2024-09-10 19:09:16.72711138 +0000 UTC m=+0.781406867 container died 2985422a0f6516b947537a145f413255f05d9050d4761ecc38f814815125f95e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mon-c, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-10T19:09:17.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:17 smithi047 podman[81986]: 2024-09-10 19:09:17.098661599 +0000 UTC m=+1.152957093 container cleanup 2985422a0f6516b947537a145f413255f05d9050d4761ecc38f814815125f95e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mon-c, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-10T19:09:17.290 INFO:journalctl@ceph.mon.c.smithi047.stdout:Sep 10 19:09:17 smithi047 bash[81986]: ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mon-c 2024-09-10T19:09:17.431 DEBUG:teuthology.orchestra.run.smithi047:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968@mon.c.service' 2024-09-10T19:09:17.471 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T19:09:17.471 INFO:tasks.cephadm.mon.b:Stopped mon.c 2024-09-10T19:09:17.471 INFO:tasks.cephadm.mon.b:Stopping mon.b... 2024-09-10T19:09:17.472 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968@mon.b 2024-09-10T19:09:17.759 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:17 smithi149 systemd[1]: Stopping Ceph mon.b for d8563d7e-6fa5-11ef-bcea-c7b262605968... 2024-09-10T19:09:18.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:17 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mon-b[33226]: 2024-09-10T19:09:17.757+0000 7ff49ce0f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T19:09:18.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:17 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mon-b[33226]: 2024-09-10T19:09:17.757+0000 7ff49ce0f640 -1 mon.b@1(peon) e3 *** Got Signal Terminated *** 2024-09-10T19:09:18.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:17 smithi149 podman[70591]: 2024-09-10 19:09:17.880244403 +0000 UTC m=+0.207054599 container died fb64053d26660f7d8ccf6937602dafd33b7b9dd44b349546da8d12af455c6280 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mon-b, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-10T19:09:18.540 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:18 smithi149 podman[70591]: 2024-09-10 19:09:18.250007216 +0000 UTC m=+0.576817401 container cleanup fb64053d26660f7d8ccf6937602dafd33b7b9dd44b349546da8d12af455c6280 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T19:09:18.540 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:18 smithi149 bash[70591]: ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mon-b 2024-09-10T19:09:18.805 INFO:journalctl@ceph.iscsi.iscsi.a.smithi149.stdout:Sep 10 19:09:18 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-iscsi-iscsi-a[60538]: debug there is no tcmu-runner data available 2024-09-10T19:09:18.805 INFO:journalctl@ceph.mon.b.smithi149.stdout:Sep 10 19:09:18 smithi149 podman[70603]: 2024-09-10 19:09:18.626731547 +0000 UTC m=+0.744263932 container remove fb64053d26660f7d8ccf6937602dafd33b7b9dd44b349546da8d12af455c6280 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mon-b, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T19:09:18.823 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968@mon.b.service' 2024-09-10T19:09:18.900 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T19:09:18.900 INFO:tasks.cephadm.mon.b:Stopped mon.b 2024-09-10T19:09:18.901 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-10T19:09:18.901 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968@mgr.y 2024-09-10T19:09:19.228 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:18 smithi047 systemd[1]: Stopping Ceph mgr.y for d8563d7e-6fa5-11ef-bcea-c7b262605968... 2024-09-10T19:09:19.539 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:19 smithi047 podman[82159]: 2024-09-10 19:09:19.234155451 +0000 UTC m=+0.133563936 container died 5f1c4720477f5b16c9491f1b52ebc7d01a326deb2ee55327457075b5d3ff8d2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T19:09:19.879 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:19 smithi047 podman[82159]: 2024-09-10 19:09:19.66090998 +0000 UTC m=+0.560318464 container cleanup 5f1c4720477f5b16c9491f1b52ebc7d01a326deb2ee55327457075b5d3ff8d2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0) 2024-09-10T19:09:19.880 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:19 smithi047 bash[82159]: ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y 2024-09-10T19:09:20.142 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 19:09:19 smithi047 podman[82172]: 2024-09-10 19:09:19.878919834 +0000 UTC m=+0.642912455 container remove 5f1c4720477f5b16c9491f1b52ebc7d01a326deb2ee55327457075b5d3ff8d2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-y, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-10T19:09:20.146 DEBUG:teuthology.orchestra.run.smithi047:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968@mgr.y.service' 2024-09-10T19:09:20.183 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T19:09:20.183 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-10T19:09:20.183 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-10T19:09:20.183 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968@mgr.x 2024-09-10T19:09:20.493 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:09:20 smithi149 systemd[1]: Stopping Ceph mgr.x for d8563d7e-6fa5-11ef-bcea-c7b262605968... 2024-09-10T19:09:20.752 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:09:20 smithi149 podman[70713]: 2024-09-10 19:09:20.492461111 +0000 UTC m=+0.112806242 container died 4a344cd2abe1ccd59b148896058d6307a0b459b13e17726f8ee7e7d6d6e14525 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-10T19:09:21.093 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:09:20 smithi149 podman[70713]: 2024-09-10 19:09:20.913618296 +0000 UTC m=+0.533963426 container cleanup 4a344cd2abe1ccd59b148896058d6307a0b459b13e17726f8ee7e7d6d6e14525 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True) 2024-09-10T19:09:21.093 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:09:20 smithi149 bash[70713]: ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x 2024-09-10T19:09:21.423 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:09:21 smithi149 podman[70728]: 2024-09-10 19:09:21.132220222 +0000 UTC m=+0.634001515 container remove 4a344cd2abe1ccd59b148896058d6307a0b459b13e17726f8ee7e7d6d6e14525 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-mgr-x, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.vendor=CentOS) 2024-09-10T19:09:21.423 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Sep 10 19:09:21 smithi149 systemd[1]: ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968@mgr.x.service: Main process exited, code=exited, status=143/n/a 2024-09-10T19:09:21.439 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968@mgr.x.service' 2024-09-10T19:09:21.476 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T19:09:21.476 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-10T19:09:21.476 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-10T19:09:21.476 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968@osd.0 2024-09-10T19:09:21.789 INFO:journalctl@ceph.osd.0.smithi047.stdout:Sep 10 19:09:21 smithi047 systemd[1]: Stopping Ceph osd.0 for d8563d7e-6fa5-11ef-bcea-c7b262605968... 2024-09-10T19:09:22.292 INFO:journalctl@ceph.osd.0.smithi047.stdout:Sep 10 19:09:21 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-0[43856]: 2024-09-10T19:09:21.797+0000 7fb938a9e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T19:09:22.292 INFO:journalctl@ceph.osd.0.smithi047.stdout:Sep 10 19:09:21 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-0[43856]: 2024-09-10T19:09:21.797+0000 7fb938a9e640 -1 osd.0 67 *** Got signal Terminated *** 2024-09-10T19:09:22.292 INFO:journalctl@ceph.osd.0.smithi047.stdout:Sep 10 19:09:21 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-0[43856]: 2024-09-10T19:09:21.797+0000 7fb938a9e640 -1 osd.0 67 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T19:09:27.640 INFO:journalctl@ceph.osd.0.smithi047.stdout:Sep 10 19:09:27 smithi047 podman[82280]: 2024-09-10 19:09:27.346317668 +0000 UTC m=+5.673574509 container died ee69fb15d47ef64f3362000b7ff4f56ba6121c6d70c6d225262b111b00217728 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-0, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-10T19:09:28.039 INFO:journalctl@ceph.osd.0.smithi047.stdout:Sep 10 19:09:27 smithi047 podman[82280]: 2024-09-10 19:09:27.63961143 +0000 UTC m=+5.966868270 container cleanup ee69fb15d47ef64f3362000b7ff4f56ba6121c6d70c6d225262b111b00217728 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-0, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-10T19:09:28.040 INFO:journalctl@ceph.osd.0.smithi047.stdout:Sep 10 19:09:27 smithi047 bash[82280]: ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-0 2024-09-10T19:09:28.040 INFO:journalctl@ceph.osd.0.smithi047.stdout:Sep 10 19:09:27 smithi047 podman[82295]: 2024-09-10 19:09:27.840711957 +0000 UTC m=+0.490757279 container remove ee69fb15d47ef64f3362000b7ff4f56ba6121c6d70c6d225262b111b00217728 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-0, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-10T19:09:28.453 INFO:journalctl@ceph.osd.0.smithi047.stdout:Sep 10 19:09:28 smithi047 podman[82384]: 2024-09-10 19:09:28.062203347 +0000 UTC m=+0.016056851 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16 2024-09-10T19:09:28.453 INFO:journalctl@ceph.osd.0.smithi047.stdout:Sep 10 19:09:28 smithi047 podman[82384]: 2024-09-10 19:09:28.191426438 +0000 UTC m=+0.145279940 container create 86287b094043fa034d0191e0e6b02ceead8af7ea8af275f2a1122a50b2e7834e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T19:09:28.453 INFO:journalctl@ceph.osd.0.smithi047.stdout:Sep 10 19:09:28 smithi047 podman[82384]: 2024-09-10 19:09:28.377228627 +0000 UTC m=+0.331082129 container init 86287b094043fa034d0191e0e6b02ceead8af7ea8af275f2a1122a50b2e7834e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T19:09:28.453 INFO:journalctl@ceph.osd.0.smithi047.stdout:Sep 10 19:09:28 smithi047 podman[82384]: 2024-09-10 19:09:28.380750333 +0000 UTC m=+0.334603835 container start 86287b094043fa034d0191e0e6b02ceead8af7ea8af275f2a1122a50b2e7834e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-0-deactivate, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-10T19:09:28.453 INFO:journalctl@ceph.osd.0.smithi047.stdout:Sep 10 19:09:28 smithi047 podman[82384]: 2024-09-10 19:09:28.451867164 +0000 UTC m=+0.405720675 container attach 86287b094043fa034d0191e0e6b02ceead8af7ea8af275f2a1122a50b2e7834e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-10T19:09:28.790 INFO:journalctl@ceph.osd.0.smithi047.stdout:Sep 10 19:09:28 smithi047 podman[82384]: 2024-09-10 19:09:28.558814411 +0000 UTC m=+0.512667913 container died 86287b094043fa034d0191e0e6b02ceead8af7ea8af275f2a1122a50b2e7834e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-10T19:09:29.196 DEBUG:teuthology.orchestra.run.smithi047:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968@osd.0.service' 2024-09-10T19:09:29.242 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T19:09:29.242 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-10T19:09:29.242 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-10T19:09:29.242 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968@osd.1 2024-09-10T19:09:29.535 INFO:journalctl@ceph.osd.1.smithi047.stdout:Sep 10 19:09:29 smithi047 systemd[1]: Stopping Ceph osd.1 for d8563d7e-6fa5-11ef-bcea-c7b262605968... 2024-09-10T19:09:29.789 INFO:journalctl@ceph.osd.1.smithi047.stdout:Sep 10 19:09:29 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-1[49392]: 2024-09-10T19:09:29.533+0000 7f4b0a681640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T19:09:29.790 INFO:journalctl@ceph.osd.1.smithi047.stdout:Sep 10 19:09:29 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-1[49392]: 2024-09-10T19:09:29.533+0000 7f4b0a681640 -1 osd.1 67 *** Got signal Terminated *** 2024-09-10T19:09:29.790 INFO:journalctl@ceph.osd.1.smithi047.stdout:Sep 10 19:09:29 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-1[49392]: 2024-09-10T19:09:29.533+0000 7f4b0a681640 -1 osd.1 67 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T19:09:35.209 INFO:journalctl@ceph.osd.1.smithi047.stdout:Sep 10 19:09:34 smithi047 podman[82489]: 2024-09-10 19:09:34.910839378 +0000 UTC m=+5.466789553 container died a530ddf7d75304dc9be1eda0288ed8974c492bc9702f7ebc6de2566340033959 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-1, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-10T19:09:35.485 INFO:journalctl@ceph.osd.1.smithi047.stdout:Sep 10 19:09:35 smithi047 podman[82489]: 2024-09-10 19:09:35.291221398 +0000 UTC m=+5.847171575 container cleanup a530ddf7d75304dc9be1eda0288ed8974c492bc9702f7ebc6de2566340033959 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-1, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T19:09:35.485 INFO:journalctl@ceph.osd.1.smithi047.stdout:Sep 10 19:09:35 smithi047 bash[82489]: ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-1 2024-09-10T19:09:35.789 INFO:journalctl@ceph.osd.1.smithi047.stdout:Sep 10 19:09:35 smithi047 podman[82504]: 2024-09-10 19:09:35.48412136 +0000 UTC m=+0.567009888 container remove a530ddf7d75304dc9be1eda0288ed8974c492bc9702f7ebc6de2566340033959 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-1, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True) 2024-09-10T19:09:36.222 INFO:journalctl@ceph.osd.1.smithi047.stdout:Sep 10 19:09:35 smithi047 podman[82590]: 2024-09-10 19:09:35.816243128 +0000 UTC m=+0.016448041 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16 2024-09-10T19:09:36.222 INFO:journalctl@ceph.osd.1.smithi047.stdout:Sep 10 19:09:35 smithi047 podman[82590]: 2024-09-10 19:09:35.926570001 +0000 UTC m=+0.126774913 container create 4a760402276fa83fb7aa7e9622918d9403665e536782e4bc0fe358172ba494e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-1-deactivate, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T19:09:36.222 INFO:journalctl@ceph.osd.1.smithi047.stdout:Sep 10 19:09:36 smithi047 podman[82590]: 2024-09-10 19:09:36.120569403 +0000 UTC m=+0.320774316 container init 4a760402276fa83fb7aa7e9622918d9403665e536782e4bc0fe358172ba494e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-10T19:09:36.222 INFO:journalctl@ceph.osd.1.smithi047.stdout:Sep 10 19:09:36 smithi047 podman[82590]: 2024-09-10 19:09:36.123906823 +0000 UTC m=+0.324111734 container start 4a760402276fa83fb7aa7e9622918d9403665e536782e4bc0fe358172ba494e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-1-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS) 2024-09-10T19:09:36.222 INFO:journalctl@ceph.osd.1.smithi047.stdout:Sep 10 19:09:36 smithi047 podman[82590]: 2024-09-10 19:09:36.22085502 +0000 UTC m=+0.421059957 container attach 4a760402276fa83fb7aa7e9622918d9403665e536782e4bc0fe358172ba494e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T19:09:36.539 INFO:journalctl@ceph.osd.1.smithi047.stdout:Sep 10 19:09:36 smithi047 podman[82590]: 2024-09-10 19:09:36.307872208 +0000 UTC m=+0.508077118 container died 4a760402276fa83fb7aa7e9622918d9403665e536782e4bc0fe358172ba494e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-10T19:09:36.852 DEBUG:teuthology.orchestra.run.smithi047:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968@osd.1.service' 2024-09-10T19:09:36.880 INFO:journalctl@ceph.osd.1.smithi047.stdout:Sep 10 19:09:36 smithi047 podman[82613]: 2024-09-10 19:09:36.832340001 +0000 UTC m=+0.517664335 container remove 4a760402276fa83fb7aa7e9622918d9403665e536782e4bc0fe358172ba494e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-10T19:09:36.880 INFO:journalctl@ceph.osd.1.smithi047.stdout:Sep 10 19:09:36 smithi047 systemd[1]: ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968@osd.1.service: Deactivated successfully. 2024-09-10T19:09:36.880 INFO:journalctl@ceph.osd.1.smithi047.stdout:Sep 10 19:09:36 smithi047 systemd[1]: Stopped Ceph osd.1 for d8563d7e-6fa5-11ef-bcea-c7b262605968. 2024-09-10T19:09:36.880 INFO:journalctl@ceph.osd.1.smithi047.stdout:Sep 10 19:09:36 smithi047 systemd[1]: ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968@osd.1.service: Consumed 6.757s CPU time. 2024-09-10T19:09:36.898 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T19:09:36.898 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-10T19:09:36.899 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-10T19:09:36.899 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968@osd.2 2024-09-10T19:09:37.245 INFO:journalctl@ceph.osd.2.smithi047.stdout:Sep 10 19:09:36 smithi047 systemd[1]: Stopping Ceph osd.2 for d8563d7e-6fa5-11ef-bcea-c7b262605968... 2024-09-10T19:09:37.539 INFO:journalctl@ceph.osd.2.smithi047.stdout:Sep 10 19:09:37 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-2[54962]: 2024-09-10T19:09:37.243+0000 7f52148b9640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T19:09:37.539 INFO:journalctl@ceph.osd.2.smithi047.stdout:Sep 10 19:09:37 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-2[54962]: 2024-09-10T19:09:37.243+0000 7f52148b9640 -1 osd.2 67 *** Got signal Terminated *** 2024-09-10T19:09:37.540 INFO:journalctl@ceph.osd.2.smithi047.stdout:Sep 10 19:09:37 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-2[54962]: 2024-09-10T19:09:37.243+0000 7f52148b9640 -1 osd.2 67 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T19:09:42.981 INFO:journalctl@ceph.osd.2.smithi047.stdout:Sep 10 19:09:42 smithi047 podman[82693]: 2024-09-10 19:09:42.693136445 +0000 UTC m=+5.565896324 container died 48c3b0129ae291ff8fa009004fea7dc6a86371412eb170b5d774990eb7ecc899 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-10T19:09:43.290 INFO:journalctl@ceph.osd.2.smithi047.stdout:Sep 10 19:09:43 smithi047 podman[82693]: 2024-09-10 19:09:43.084877641 +0000 UTC m=+5.957637519 container cleanup 48c3b0129ae291ff8fa009004fea7dc6a86371412eb170b5d774990eb7ecc899 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-2, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-10T19:09:43.290 INFO:journalctl@ceph.osd.2.smithi047.stdout:Sep 10 19:09:43 smithi047 bash[82693]: ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-2 2024-09-10T19:09:43.654 INFO:journalctl@ceph.osd.2.smithi047.stdout:Sep 10 19:09:43 smithi047 podman[82720]: 2024-09-10 19:09:43.32465586 +0000 UTC m=+0.632935385 container remove 48c3b0129ae291ff8fa009004fea7dc6a86371412eb170b5d774990eb7ecc899 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731) 2024-09-10T19:09:43.957 INFO:journalctl@ceph.osd.2.smithi047.stdout:Sep 10 19:09:43 smithi047 podman[82808]: 2024-09-10 19:09:43.653828208 +0000 UTC m=+0.107995369 container create 6e89dfd921db325452589f49d2e0c950e8e754aced3030d46073d7eb67e48604 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-10T19:09:43.957 INFO:journalctl@ceph.osd.2.smithi047.stdout:Sep 10 19:09:43 smithi047 podman[82808]: 2024-09-10 19:09:43.562500615 +0000 UTC m=+0.016667773 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16 2024-09-10T19:09:43.957 INFO:journalctl@ceph.osd.2.smithi047.stdout:Sep 10 19:09:43 smithi047 podman[82808]: 2024-09-10 19:09:43.872771897 +0000 UTC m=+0.326939056 container init 6e89dfd921db325452589f49d2e0c950e8e754aced3030d46073d7eb67e48604 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-10T19:09:43.957 INFO:journalctl@ceph.osd.2.smithi047.stdout:Sep 10 19:09:43 smithi047 podman[82808]: 2024-09-10 19:09:43.876220393 +0000 UTC m=+0.330387552 container start 6e89dfd921db325452589f49d2e0c950e8e754aced3030d46073d7eb67e48604 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, maintainer=Guillaume Abrioux ) 2024-09-10T19:09:43.957 INFO:journalctl@ceph.osd.2.smithi047.stdout:Sep 10 19:09:43 smithi047 podman[82808]: 2024-09-10 19:09:43.955976078 +0000 UTC m=+0.410143250 container attach 6e89dfd921db325452589f49d2e0c950e8e754aced3030d46073d7eb67e48604 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-10T19:09:44.289 INFO:journalctl@ceph.osd.2.smithi047.stdout:Sep 10 19:09:44 smithi047 podman[82808]: 2024-09-10 19:09:44.054282072 +0000 UTC m=+0.508449231 container died 6e89dfd921db325452589f49d2e0c950e8e754aced3030d46073d7eb67e48604 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-10T19:09:44.627 INFO:journalctl@ceph.osd.2.smithi047.stdout:Sep 10 19:09:44 smithi047 podman[82831]: 2024-09-10 19:09:44.609195329 +0000 UTC m=+0.548571480 container remove 6e89dfd921db325452589f49d2e0c950e8e754aced3030d46073d7eb67e48604 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T19:09:44.632 DEBUG:teuthology.orchestra.run.smithi047:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968@osd.2.service' 2024-09-10T19:09:44.673 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T19:09:44.673 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-10T19:09:44.673 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-10T19:09:44.673 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968@osd.3 2024-09-10T19:09:45.010 INFO:journalctl@ceph.osd.3.smithi047.stdout:Sep 10 19:09:44 smithi047 systemd[1]: Stopping Ceph osd.3 for d8563d7e-6fa5-11ef-bcea-c7b262605968... 2024-09-10T19:09:45.289 INFO:journalctl@ceph.osd.3.smithi047.stdout:Sep 10 19:09:45 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-3[60718]: 2024-09-10T19:09:45.007+0000 7f0666c48640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T19:09:45.290 INFO:journalctl@ceph.osd.3.smithi047.stdout:Sep 10 19:09:45 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-3[60718]: 2024-09-10T19:09:45.007+0000 7f0666c48640 -1 osd.3 67 *** Got signal Terminated *** 2024-09-10T19:09:45.290 INFO:journalctl@ceph.osd.3.smithi047.stdout:Sep 10 19:09:45 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-3[60718]: 2024-09-10T19:09:45.007+0000 7f0666c48640 -1 osd.3 67 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T19:09:48.343 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:09:47 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7[55520]: 2024-09-10T19:09:47.864+0000 7f042d57e640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.47:6803 osd.0 since back 2024-09-10T19:09:25.558659+0000 front 2024-09-10T19:09:25.558741+0000 (oldest deadline 2024-09-10T19:09:47.858457+0000) 2024-09-10T19:09:48.843 INFO:journalctl@ceph.osd.6.smithi149.stdout:Sep 10 19:09:48 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-6[49840]: 2024-09-10T19:09:48.434+0000 7f20c5917640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.47:6803 osd.0 since back 2024-09-10T19:09:25.320938+0000 front 2024-09-10T19:09:25.320862+0000 (oldest deadline 2024-09-10T19:09:47.620834+0000) 2024-09-10T19:09:49.343 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:09:48 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7[55520]: 2024-09-10T19:09:48.903+0000 7f042d57e640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.47:6803 osd.0 since back 2024-09-10T19:09:25.558659+0000 front 2024-09-10T19:09:25.558741+0000 (oldest deadline 2024-09-10T19:09:47.858457+0000) 2024-09-10T19:09:49.843 INFO:journalctl@ceph.osd.6.smithi149.stdout:Sep 10 19:09:49 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-6[49840]: 2024-09-10T19:09:49.445+0000 7f20c5917640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.47:6803 osd.0 since back 2024-09-10T19:09:25.320938+0000 front 2024-09-10T19:09:25.320862+0000 (oldest deadline 2024-09-10T19:09:47.620834+0000) 2024-09-10T19:09:50.343 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:09:49 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7[55520]: 2024-09-10T19:09:49.937+0000 7f042d57e640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.47:6803 osd.0 since back 2024-09-10T19:09:25.558659+0000 front 2024-09-10T19:09:25.558741+0000 (oldest deadline 2024-09-10T19:09:47.858457+0000) 2024-09-10T19:09:50.613 INFO:journalctl@ceph.osd.5.smithi149.stdout:Sep 10 19:09:50 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-5[44040]: 2024-09-10T19:09:50.399+0000 7fd4a9b91640 -1 osd.5 67 heartbeat_check: no reply from 172.21.15.47:6803 osd.0 since back 2024-09-10T19:09:22.439498+0000 front 2024-09-10T19:09:22.439454+0000 (oldest deadline 2024-09-10T19:09:50.040068+0000) 2024-09-10T19:09:50.613 INFO:journalctl@ceph.osd.6.smithi149.stdout:Sep 10 19:09:50 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-6[49840]: 2024-09-10T19:09:50.422+0000 7f20c5917640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.47:6803 osd.0 since back 2024-09-10T19:09:25.320938+0000 front 2024-09-10T19:09:25.320862+0000 (oldest deadline 2024-09-10T19:09:47.620834+0000) 2024-09-10T19:09:50.756 INFO:journalctl@ceph.osd.3.smithi047.stdout:Sep 10 19:09:50 smithi047 podman[82911]: 2024-09-10 19:09:50.456775621 +0000 UTC m=+5.560072074 container died 932ab7add1ec0445cd599869bee498ab4c4e6412e8eb0a0a679b5e81802892d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-3, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-10T19:09:51.039 INFO:journalctl@ceph.osd.3.smithi047.stdout:Sep 10 19:09:50 smithi047 podman[82911]: 2024-09-10 19:09:50.828371416 +0000 UTC m=+5.931667863 container cleanup 932ab7add1ec0445cd599869bee498ab4c4e6412e8eb0a0a679b5e81802892d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-3, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-10T19:09:51.040 INFO:journalctl@ceph.osd.3.smithi047.stdout:Sep 10 19:09:50 smithi047 bash[82911]: ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-3 2024-09-10T19:09:51.343 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:09:50 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7[55520]: 2024-09-10T19:09:50.957+0000 7f042d57e640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.47:6803 osd.0 since back 2024-09-10T19:09:25.558659+0000 front 2024-09-10T19:09:25.558741+0000 (oldest deadline 2024-09-10T19:09:47.858457+0000) 2024-09-10T19:09:51.372 INFO:journalctl@ceph.osd.3.smithi047.stdout:Sep 10 19:09:51 smithi047 podman[82929]: 2024-09-10 19:09:51.046076272 +0000 UTC m=+0.579316913 container remove 932ab7add1ec0445cd599869bee498ab4c4e6412e8eb0a0a679b5e81802892d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T19:09:51.630 INFO:journalctl@ceph.osd.3.smithi047.stdout:Sep 10 19:09:51 smithi047 podman[83018]: 2024-09-10 19:09:51.371762038 +0000 UTC m=+0.108902999 container create 34c16d77da1b3e38f7e73f3b32ef8e913c0bbb6567bf24cb048dc1825092725f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-3-deactivate, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-10T19:09:51.630 INFO:journalctl@ceph.osd.3.smithi047.stdout:Sep 10 19:09:51 smithi047 podman[83018]: 2024-09-10 19:09:51.279275278 +0000 UTC m=+0.016416242 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16 2024-09-10T19:09:51.675 INFO:journalctl@ceph.osd.5.smithi149.stdout:Sep 10 19:09:51 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-5[44040]: 2024-09-10T19:09:51.363+0000 7fd4a9b91640 -1 osd.5 67 heartbeat_check: no reply from 172.21.15.47:6803 osd.0 since back 2024-09-10T19:09:22.439498+0000 front 2024-09-10T19:09:22.439454+0000 (oldest deadline 2024-09-10T19:09:50.040068+0000) 2024-09-10T19:09:51.676 INFO:journalctl@ceph.osd.6.smithi149.stdout:Sep 10 19:09:51 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-6[49840]: 2024-09-10T19:09:51.381+0000 7f20c5917640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.47:6803 osd.0 since back 2024-09-10T19:09:25.320938+0000 front 2024-09-10T19:09:25.320862+0000 (oldest deadline 2024-09-10T19:09:47.620834+0000) 2024-09-10T19:09:51.935 INFO:journalctl@ceph.osd.4.smithi149.stdout:Sep 10 19:09:51 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-4[38043]: 2024-09-10T19:09:51.673+0000 7f4ea4d12640 -1 osd.4 67 heartbeat_check: no reply from 172.21.15.47:6803 osd.0 since back 2024-09-10T19:09:25.694843+0000 front 2024-09-10T19:09:25.694747+0000 (oldest deadline 2024-09-10T19:09:51.594652+0000) 2024-09-10T19:09:52.040 INFO:journalctl@ceph.osd.3.smithi047.stdout:Sep 10 19:09:51 smithi047 podman[83018]: 2024-09-10 19:09:51.774256455 +0000 UTC m=+0.511397426 container init 34c16d77da1b3e38f7e73f3b32ef8e913c0bbb6567bf24cb048dc1825092725f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-3-deactivate, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-10T19:09:52.040 INFO:journalctl@ceph.osd.3.smithi047.stdout:Sep 10 19:09:51 smithi047 podman[83018]: 2024-09-10 19:09:51.777614983 +0000 UTC m=+0.514755944 container start 34c16d77da1b3e38f7e73f3b32ef8e913c0bbb6567bf24cb048dc1825092725f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-10T19:09:52.040 INFO:journalctl@ceph.osd.3.smithi047.stdout:Sep 10 19:09:51 smithi047 podman[83018]: 2024-09-10 19:09:51.857252054 +0000 UTC m=+0.594393024 container attach 34c16d77da1b3e38f7e73f3b32ef8e913c0bbb6567bf24cb048dc1825092725f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-10T19:09:52.040 INFO:journalctl@ceph.osd.3.smithi047.stdout:Sep 10 19:09:51 smithi047 podman[83018]: 2024-09-10 19:09:51.959802229 +0000 UTC m=+0.696943201 container died 34c16d77da1b3e38f7e73f3b32ef8e913c0bbb6567bf24cb048dc1825092725f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-10T19:09:52.342 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:09:51 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7[55520]: 2024-09-10T19:09:51.933+0000 7f042d57e640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.47:6803 osd.0 since back 2024-09-10T19:09:25.558659+0000 front 2024-09-10T19:09:25.558741+0000 (oldest deadline 2024-09-10T19:09:47.858457+0000) 2024-09-10T19:09:52.496 INFO:journalctl@ceph.osd.3.smithi047.stdout:Sep 10 19:09:52 smithi047 podman[83041]: 2024-09-10 19:09:52.4935154 +0000 UTC m=+0.527656312 container remove 34c16d77da1b3e38f7e73f3b32ef8e913c0bbb6567bf24cb048dc1825092725f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-10T19:09:52.511 DEBUG:teuthology.orchestra.run.smithi047:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968@osd.3.service' 2024-09-10T19:09:52.552 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T19:09:52.552 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-10T19:09:52.553 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-10T19:09:52.553 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968@osd.4 2024-09-10T19:09:52.695 INFO:journalctl@ceph.osd.4.smithi149.stdout:Sep 10 19:09:52 smithi149 systemd[1]: Stopping Ceph osd.4 for d8563d7e-6fa5-11ef-bcea-c7b262605968... 2024-09-10T19:09:52.695 INFO:journalctl@ceph.osd.5.smithi149.stdout:Sep 10 19:09:52 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-5[44040]: 2024-09-10T19:09:52.364+0000 7fd4a9b91640 -1 osd.5 67 heartbeat_check: no reply from 172.21.15.47:6803 osd.0 since back 2024-09-10T19:09:22.439498+0000 front 2024-09-10T19:09:22.439454+0000 (oldest deadline 2024-09-10T19:09:50.040068+0000) 2024-09-10T19:09:52.696 INFO:journalctl@ceph.osd.6.smithi149.stdout:Sep 10 19:09:52 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-6[49840]: 2024-09-10T19:09:52.393+0000 7f20c5917640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.47:6803 osd.0 since back 2024-09-10T19:09:25.320938+0000 front 2024-09-10T19:09:25.320862+0000 (oldest deadline 2024-09-10T19:09:47.620834+0000) 2024-09-10T19:09:53.093 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:09:52 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7[55520]: 2024-09-10T19:09:52.908+0000 7f042d57e640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.47:6803 osd.0 since back 2024-09-10T19:09:25.558659+0000 front 2024-09-10T19:09:25.558741+0000 (oldest deadline 2024-09-10T19:09:47.858457+0000) 2024-09-10T19:09:53.093 INFO:journalctl@ceph.osd.4.smithi149.stdout:Sep 10 19:09:52 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-4[38043]: 2024-09-10T19:09:52.693+0000 7f4ea4d12640 -1 osd.4 67 heartbeat_check: no reply from 172.21.15.47:6803 osd.0 since back 2024-09-10T19:09:25.694843+0000 front 2024-09-10T19:09:25.694747+0000 (oldest deadline 2024-09-10T19:09:51.594652+0000) 2024-09-10T19:09:53.093 INFO:journalctl@ceph.osd.4.smithi149.stdout:Sep 10 19:09:52 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-4[38043]: 2024-09-10T19:09:52.859+0000 7f4eab700640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T19:09:53.093 INFO:journalctl@ceph.osd.4.smithi149.stdout:Sep 10 19:09:52 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-4[38043]: 2024-09-10T19:09:52.859+0000 7f4eab700640 -1 osd.4 67 *** Got signal Terminated *** 2024-09-10T19:09:53.093 INFO:journalctl@ceph.osd.4.smithi149.stdout:Sep 10 19:09:52 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-4[38043]: 2024-09-10T19:09:52.859+0000 7f4eab700640 -1 osd.4 67 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T19:09:53.675 INFO:journalctl@ceph.osd.5.smithi149.stdout:Sep 10 19:09:53 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-5[44040]: 2024-09-10T19:09:53.353+0000 7fd4a9b91640 -1 osd.5 67 heartbeat_check: no reply from 172.21.15.47:6803 osd.0 since back 2024-09-10T19:09:22.439498+0000 front 2024-09-10T19:09:22.439454+0000 (oldest deadline 2024-09-10T19:09:50.040068+0000) 2024-09-10T19:09:53.675 INFO:journalctl@ceph.osd.6.smithi149.stdout:Sep 10 19:09:53 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-6[49840]: 2024-09-10T19:09:53.440+0000 7f20c5917640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.47:6803 osd.0 since back 2024-09-10T19:09:25.320938+0000 front 2024-09-10T19:09:25.320862+0000 (oldest deadline 2024-09-10T19:09:47.620834+0000) 2024-09-10T19:09:53.927 INFO:journalctl@ceph.osd.4.smithi149.stdout:Sep 10 19:09:53 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-4[38043]: 2024-09-10T19:09:53.673+0000 7f4ea4d12640 -1 osd.4 67 heartbeat_check: no reply from 172.21.15.47:6803 osd.0 since back 2024-09-10T19:09:25.694843+0000 front 2024-09-10T19:09:25.694747+0000 (oldest deadline 2024-09-10T19:09:51.594652+0000) 2024-09-10T19:09:54.343 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:09:53 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7[55520]: 2024-09-10T19:09:53.926+0000 7f042d57e640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.47:6803 osd.0 since back 2024-09-10T19:09:25.558659+0000 front 2024-09-10T19:09:25.558741+0000 (oldest deadline 2024-09-10T19:09:47.858457+0000) 2024-09-10T19:09:54.658 INFO:journalctl@ceph.osd.5.smithi149.stdout:Sep 10 19:09:54 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-5[44040]: 2024-09-10T19:09:54.358+0000 7fd4a9b91640 -1 osd.5 67 heartbeat_check: no reply from 172.21.15.47:6803 osd.0 since back 2024-09-10T19:09:22.439498+0000 front 2024-09-10T19:09:22.439454+0000 (oldest deadline 2024-09-10T19:09:50.040068+0000) 2024-09-10T19:09:54.659 INFO:journalctl@ceph.osd.6.smithi149.stdout:Sep 10 19:09:54 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-6[49840]: 2024-09-10T19:09:54.395+0000 7f20c5917640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.47:6803 osd.0 since back 2024-09-10T19:09:25.320938+0000 front 2024-09-10T19:09:25.320862+0000 (oldest deadline 2024-09-10T19:09:47.620834+0000) 2024-09-10T19:09:54.963 INFO:journalctl@ceph.osd.4.smithi149.stdout:Sep 10 19:09:54 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-4[38043]: 2024-09-10T19:09:54.657+0000 7f4ea4d12640 -1 osd.4 67 heartbeat_check: no reply from 172.21.15.47:6803 osd.0 since back 2024-09-10T19:09:25.694843+0000 front 2024-09-10T19:09:25.694747+0000 (oldest deadline 2024-09-10T19:09:51.594652+0000) 2024-09-10T19:09:55.333 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:09:54 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7[55520]: 2024-09-10T19:09:54.961+0000 7f042d57e640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.47:6803 osd.0 since back 2024-09-10T19:09:25.558659+0000 front 2024-09-10T19:09:25.558741+0000 (oldest deadline 2024-09-10T19:09:47.858457+0000) 2024-09-10T19:09:55.593 INFO:journalctl@ceph.osd.5.smithi149.stdout:Sep 10 19:09:55 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-5[44040]: 2024-09-10T19:09:55.331+0000 7fd4a9b91640 -1 osd.5 67 heartbeat_check: no reply from 172.21.15.47:6803 osd.0 since back 2024-09-10T19:09:22.439498+0000 front 2024-09-10T19:09:22.439454+0000 (oldest deadline 2024-09-10T19:09:50.040068+0000) 2024-09-10T19:09:55.593 INFO:journalctl@ceph.osd.6.smithi149.stdout:Sep 10 19:09:55 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-6[49840]: 2024-09-10T19:09:55.420+0000 7f20c5917640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.47:6803 osd.0 since back 2024-09-10T19:09:25.320938+0000 front 2024-09-10T19:09:25.320862+0000 (oldest deadline 2024-09-10T19:09:47.620834+0000) 2024-09-10T19:09:56.002 INFO:journalctl@ceph.osd.4.smithi149.stdout:Sep 10 19:09:55 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-4[38043]: 2024-09-10T19:09:55.695+0000 7f4ea4d12640 -1 osd.4 67 heartbeat_check: no reply from 172.21.15.47:6803 osd.0 since back 2024-09-10T19:09:25.694843+0000 front 2024-09-10T19:09:25.694747+0000 (oldest deadline 2024-09-10T19:09:51.594652+0000) 2024-09-10T19:09:56.338 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:09:56 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7[55520]: 2024-09-10T19:09:56.000+0000 7f042d57e640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.47:6803 osd.0 since back 2024-09-10T19:09:25.558659+0000 front 2024-09-10T19:09:25.558741+0000 (oldest deadline 2024-09-10T19:09:47.858457+0000) 2024-09-10T19:09:56.338 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:09:56 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7[55520]: 2024-09-10T19:09:56.000+0000 7f042d57e640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.47:6807 osd.1 since back 2024-09-10T19:09:31.859874+0000 front 2024-09-10T19:09:31.859940+0000 (oldest deadline 2024-09-10T19:09:55.359754+0000) 2024-09-10T19:09:56.593 INFO:journalctl@ceph.osd.5.smithi149.stdout:Sep 10 19:09:56 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-5[44040]: 2024-09-10T19:09:56.336+0000 7fd4a9b91640 -1 osd.5 67 heartbeat_check: no reply from 172.21.15.47:6803 osd.0 since back 2024-09-10T19:09:22.439498+0000 front 2024-09-10T19:09:22.439454+0000 (oldest deadline 2024-09-10T19:09:50.040068+0000) 2024-09-10T19:09:56.593 INFO:journalctl@ceph.osd.6.smithi149.stdout:Sep 10 19:09:56 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-6[49840]: 2024-09-10T19:09:56.405+0000 7f20c5917640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.47:6803 osd.0 since back 2024-09-10T19:09:25.320938+0000 front 2024-09-10T19:09:25.320862+0000 (oldest deadline 2024-09-10T19:09:47.620834+0000) 2024-09-10T19:09:57.028 INFO:journalctl@ceph.osd.4.smithi149.stdout:Sep 10 19:09:56 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-4[38043]: 2024-09-10T19:09:56.693+0000 7f4ea4d12640 -1 osd.4 67 heartbeat_check: no reply from 172.21.15.47:6803 osd.0 since back 2024-09-10T19:09:25.694843+0000 front 2024-09-10T19:09:25.694747+0000 (oldest deadline 2024-09-10T19:09:51.594652+0000) 2024-09-10T19:09:57.301 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:09:57 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7[55520]: 2024-09-10T19:09:57.027+0000 7f042d57e640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.47:6803 osd.0 since back 2024-09-10T19:09:25.558659+0000 front 2024-09-10T19:09:25.558741+0000 (oldest deadline 2024-09-10T19:09:47.858457+0000) 2024-09-10T19:09:57.302 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:09:57 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7[55520]: 2024-09-10T19:09:57.027+0000 7f042d57e640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.47:6807 osd.1 since back 2024-09-10T19:09:31.859874+0000 front 2024-09-10T19:09:31.859940+0000 (oldest deadline 2024-09-10T19:09:55.359754+0000) 2024-09-10T19:09:57.593 INFO:journalctl@ceph.osd.6.smithi149.stdout:Sep 10 19:09:57 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-6[49840]: 2024-09-10T19:09:57.390+0000 7f20c5917640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.47:6803 osd.0 since back 2024-09-10T19:09:25.320938+0000 front 2024-09-10T19:09:25.320862+0000 (oldest deadline 2024-09-10T19:09:47.620834+0000) 2024-09-10T19:09:57.593 INFO:journalctl@ceph.osd.5.smithi149.stdout:Sep 10 19:09:57 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-5[44040]: 2024-09-10T19:09:57.300+0000 7fd4a9b91640 -1 osd.5 67 heartbeat_check: no reply from 172.21.15.47:6803 osd.0 since back 2024-09-10T19:09:22.439498+0000 front 2024-09-10T19:09:22.439454+0000 (oldest deadline 2024-09-10T19:09:50.040068+0000) 2024-09-10T19:09:57.593 INFO:journalctl@ceph.osd.5.smithi149.stdout:Sep 10 19:09:57 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-5[44040]: 2024-09-10T19:09:57.300+0000 7fd4a9b91640 -1 osd.5 67 heartbeat_check: no reply from 172.21.15.47:6807 osd.1 since back 2024-09-10T19:09:32.941381+0000 front 2024-09-10T19:09:32.941232+0000 (oldest deadline 2024-09-10T19:09:57.041381+0000) 2024-09-10T19:09:58.064 INFO:journalctl@ceph.osd.4.smithi149.stdout:Sep 10 19:09:57 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-4[38043]: 2024-09-10T19:09:57.696+0000 7f4ea4d12640 -1 osd.4 67 heartbeat_check: no reply from 172.21.15.47:6803 osd.0 since back 2024-09-10T19:09:25.694843+0000 front 2024-09-10T19:09:25.694747+0000 (oldest deadline 2024-09-10T19:09:51.594652+0000) 2024-09-10T19:09:58.064 INFO:journalctl@ceph.osd.4.smithi149.stdout:Sep 10 19:09:57 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-4[38043]: 2024-09-10T19:09:57.696+0000 7f4ea4d12640 -1 osd.4 67 heartbeat_check: no reply from 172.21.15.47:6807 osd.1 since back 2024-09-10T19:09:33.295647+0000 front 2024-09-10T19:09:33.295769+0000 (oldest deadline 2024-09-10T19:09:56.795557+0000) 2024-09-10T19:09:58.314 INFO:journalctl@ceph.osd.4.smithi149.stdout:Sep 10 19:09:58 smithi149 podman[70862]: 2024-09-10 19:09:58.256571221 +0000 UTC m=+5.508054517 container died 4d02311b87db0b64d9d29b7add46202eafdb59324dd2e7ed5b14d267cdf81cf1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-10T19:09:58.315 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:09:58 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7[55520]: 2024-09-10T19:09:58.062+0000 7f042d57e640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.47:6803 osd.0 since back 2024-09-10T19:09:25.558659+0000 front 2024-09-10T19:09:25.558741+0000 (oldest deadline 2024-09-10T19:09:47.858457+0000) 2024-09-10T19:09:58.315 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:09:58 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7[55520]: 2024-09-10T19:09:58.062+0000 7f042d57e640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.47:6807 osd.1 since back 2024-09-10T19:09:31.859874+0000 front 2024-09-10T19:09:31.859940+0000 (oldest deadline 2024-09-10T19:09:55.359754+0000) 2024-09-10T19:09:58.585 INFO:journalctl@ceph.osd.6.smithi149.stdout:Sep 10 19:09:58 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-6[49840]: 2024-09-10T19:09:58.404+0000 7f20c5917640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.47:6803 osd.0 since back 2024-09-10T19:09:25.320938+0000 front 2024-09-10T19:09:25.320862+0000 (oldest deadline 2024-09-10T19:09:47.620834+0000) 2024-09-10T19:09:58.585 INFO:journalctl@ceph.osd.5.smithi149.stdout:Sep 10 19:09:58 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-5[44040]: 2024-09-10T19:09:58.313+0000 7fd4a9b91640 -1 osd.5 67 heartbeat_check: no reply from 172.21.15.47:6803 osd.0 since back 2024-09-10T19:09:22.439498+0000 front 2024-09-10T19:09:22.439454+0000 (oldest deadline 2024-09-10T19:09:50.040068+0000) 2024-09-10T19:09:58.585 INFO:journalctl@ceph.osd.5.smithi149.stdout:Sep 10 19:09:58 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-5[44040]: 2024-09-10T19:09:58.313+0000 7fd4a9b91640 -1 osd.5 67 heartbeat_check: no reply from 172.21.15.47:6807 osd.1 since back 2024-09-10T19:09:32.941381+0000 front 2024-09-10T19:09:32.941232+0000 (oldest deadline 2024-09-10T19:09:57.041381+0000) 2024-09-10T19:09:58.843 INFO:journalctl@ceph.osd.4.smithi149.stdout:Sep 10 19:09:58 smithi149 podman[70862]: 2024-09-10 19:09:58.584314579 +0000 UTC m=+5.835797874 container cleanup 4d02311b87db0b64d9d29b7add46202eafdb59324dd2e7ed5b14d267cdf81cf1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-4, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-10T19:09:58.843 INFO:journalctl@ceph.osd.4.smithi149.stdout:Sep 10 19:09:58 smithi149 bash[70862]: ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-4 2024-09-10T19:09:58.843 INFO:journalctl@ceph.osd.4.smithi149.stdout:Sep 10 19:09:58 smithi149 podman[70879]: 2024-09-10 19:09:58.785611144 +0000 UTC m=+0.525719316 container remove 4d02311b87db0b64d9d29b7add46202eafdb59324dd2e7ed5b14d267cdf81cf1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-4, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T19:09:59.340 INFO:journalctl@ceph.osd.4.smithi149.stdout:Sep 10 19:09:59 smithi149 podman[70965]: 2024-09-10 19:09:59.097411043 +0000 UTC m=+0.016311142 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16 2024-09-10T19:09:59.340 INFO:journalctl@ceph.osd.4.smithi149.stdout:Sep 10 19:09:59 smithi149 podman[70965]: 2024-09-10 19:09:59.21992199 +0000 UTC m=+0.138822090 container create 61987a177dee01b521f62cc43e6e40571120416d0d9d69bec62f1c7d77d9c486 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_BRANCH=HEAD) 2024-09-10T19:09:59.340 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:09:59 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7[55520]: 2024-09-10T19:09:59.034+0000 7f042d57e640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.47:6803 osd.0 since back 2024-09-10T19:09:25.558659+0000 front 2024-09-10T19:09:25.558741+0000 (oldest deadline 2024-09-10T19:09:47.858457+0000) 2024-09-10T19:09:59.341 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:09:59 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7[55520]: 2024-09-10T19:09:59.034+0000 7f042d57e640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.47:6807 osd.1 since back 2024-09-10T19:09:31.859874+0000 front 2024-09-10T19:09:31.859940+0000 (oldest deadline 2024-09-10T19:09:55.359754+0000) 2024-09-10T19:09:59.593 INFO:journalctl@ceph.osd.6.smithi149.stdout:Sep 10 19:09:59 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-6[49840]: 2024-09-10T19:09:59.404+0000 7f20c5917640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.47:6803 osd.0 since back 2024-09-10T19:09:25.320938+0000 front 2024-09-10T19:09:25.320862+0000 (oldest deadline 2024-09-10T19:09:47.620834+0000) 2024-09-10T19:09:59.593 INFO:journalctl@ceph.osd.5.smithi149.stdout:Sep 10 19:09:59 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-5[44040]: 2024-09-10T19:09:59.355+0000 7fd4a9b91640 -1 osd.5 67 heartbeat_check: no reply from 172.21.15.47:6803 osd.0 since back 2024-09-10T19:09:22.439498+0000 front 2024-09-10T19:09:22.439454+0000 (oldest deadline 2024-09-10T19:09:50.040068+0000) 2024-09-10T19:09:59.593 INFO:journalctl@ceph.osd.5.smithi149.stdout:Sep 10 19:09:59 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-5[44040]: 2024-09-10T19:09:59.355+0000 7fd4a9b91640 -1 osd.5 67 heartbeat_check: no reply from 172.21.15.47:6807 osd.1 since back 2024-09-10T19:09:32.941381+0000 front 2024-09-10T19:09:32.941232+0000 (oldest deadline 2024-09-10T19:09:57.041381+0000) 2024-09-10T19:09:59.593 INFO:journalctl@ceph.osd.4.smithi149.stdout:Sep 10 19:09:59 smithi149 podman[70965]: 2024-09-10 19:09:59.447560201 +0000 UTC m=+0.366460313 container init 61987a177dee01b521f62cc43e6e40571120416d0d9d69bec62f1c7d77d9c486 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-4-deactivate, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-10T19:09:59.593 INFO:journalctl@ceph.osd.4.smithi149.stdout:Sep 10 19:09:59 smithi149 podman[70965]: 2024-09-10 19:09:59.450817395 +0000 UTC m=+0.369717495 container start 61987a177dee01b521f62cc43e6e40571120416d0d9d69bec62f1c7d77d9c486 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS) 2024-09-10T19:09:59.594 INFO:journalctl@ceph.osd.4.smithi149.stdout:Sep 10 19:09:59 smithi149 podman[70965]: 2024-09-10 19:09:59.514119733 +0000 UTC m=+0.433019844 container attach 61987a177dee01b521f62cc43e6e40571120416d0d9d69bec62f1c7d77d9c486 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-10T19:09:59.918 INFO:journalctl@ceph.osd.4.smithi149.stdout:Sep 10 19:09:59 smithi149 podman[70965]: 2024-09-10 19:09:59.633192572 +0000 UTC m=+0.552092676 container died 61987a177dee01b521f62cc43e6e40571120416d0d9d69bec62f1c7d77d9c486 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T19:10:00.213 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:10:00 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7[55520]: 2024-09-10T19:10:00.036+0000 7f042d57e640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.47:6803 osd.0 since back 2024-09-10T19:09:25.558659+0000 front 2024-09-10T19:09:25.558741+0000 (oldest deadline 2024-09-10T19:09:47.858457+0000) 2024-09-10T19:10:00.213 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:10:00 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7[55520]: 2024-09-10T19:10:00.036+0000 7f042d57e640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.47:6807 osd.1 since back 2024-09-10T19:09:31.859874+0000 front 2024-09-10T19:09:31.859940+0000 (oldest deadline 2024-09-10T19:09:55.359754+0000) 2024-09-10T19:10:00.233 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968@osd.4.service' 2024-09-10T19:10:00.278 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T19:10:00.278 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-10T19:10:00.279 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-10T19:10:00.279 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968@osd.5 2024-09-10T19:10:00.589 INFO:journalctl@ceph.osd.6.smithi149.stdout:Sep 10 19:10:00 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-6[49840]: 2024-09-10T19:10:00.426+0000 7f20c5917640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.47:6803 osd.0 since back 2024-09-10T19:09:25.320938+0000 front 2024-09-10T19:09:25.320862+0000 (oldest deadline 2024-09-10T19:09:47.620834+0000) 2024-09-10T19:10:00.590 INFO:journalctl@ceph.osd.6.smithi149.stdout:Sep 10 19:10:00 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-6[49840]: 2024-09-10T19:10:00.426+0000 7f20c5917640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.47:6807 osd.1 since back 2024-09-10T19:09:33.521625+0000 front 2024-09-10T19:09:33.521700+0000 (oldest deadline 2024-09-10T19:09:59.421548+0000) 2024-09-10T19:10:00.590 INFO:journalctl@ceph.osd.5.smithi149.stdout:Sep 10 19:10:00 smithi149 systemd[1]: Stopping Ceph osd.5 for d8563d7e-6fa5-11ef-bcea-c7b262605968... 2024-09-10T19:10:00.590 INFO:journalctl@ceph.osd.5.smithi149.stdout:Sep 10 19:10:00 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-5[44040]: 2024-09-10T19:10:00.329+0000 7fd4a9b91640 -1 osd.5 67 heartbeat_check: no reply from 172.21.15.47:6803 osd.0 since back 2024-09-10T19:09:22.439498+0000 front 2024-09-10T19:09:22.439454+0000 (oldest deadline 2024-09-10T19:09:50.040068+0000) 2024-09-10T19:10:00.590 INFO:journalctl@ceph.osd.5.smithi149.stdout:Sep 10 19:10:00 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-5[44040]: 2024-09-10T19:10:00.329+0000 7fd4a9b91640 -1 osd.5 67 heartbeat_check: no reply from 172.21.15.47:6807 osd.1 since back 2024-09-10T19:09:32.941381+0000 front 2024-09-10T19:09:32.941232+0000 (oldest deadline 2024-09-10T19:09:57.041381+0000) 2024-09-10T19:10:00.843 INFO:journalctl@ceph.osd.5.smithi149.stdout:Sep 10 19:10:00 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-5[44040]: 2024-09-10T19:10:00.588+0000 7fd4abd76640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T19:10:00.843 INFO:journalctl@ceph.osd.5.smithi149.stdout:Sep 10 19:10:00 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-5[44040]: 2024-09-10T19:10:00.588+0000 7fd4abd76640 -1 osd.5 67 *** Got signal Terminated *** 2024-09-10T19:10:00.843 INFO:journalctl@ceph.osd.5.smithi149.stdout:Sep 10 19:10:00 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-5[44040]: 2024-09-10T19:10:00.588+0000 7fd4abd76640 -1 osd.5 67 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T19:10:01.343 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:10:01 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7[55520]: 2024-09-10T19:10:01.010+0000 7f042d57e640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.47:6803 osd.0 since back 2024-09-10T19:09:25.558659+0000 front 2024-09-10T19:09:25.558741+0000 (oldest deadline 2024-09-10T19:09:47.858457+0000) 2024-09-10T19:10:01.343 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:10:01 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7[55520]: 2024-09-10T19:10:01.010+0000 7f042d57e640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.47:6807 osd.1 since back 2024-09-10T19:09:31.859874+0000 front 2024-09-10T19:09:31.859940+0000 (oldest deadline 2024-09-10T19:09:55.359754+0000) 2024-09-10T19:10:01.843 INFO:journalctl@ceph.osd.6.smithi149.stdout:Sep 10 19:10:01 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-6[49840]: 2024-09-10T19:10:01.434+0000 7f20c5917640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.47:6803 osd.0 since back 2024-09-10T19:09:25.320938+0000 front 2024-09-10T19:09:25.320862+0000 (oldest deadline 2024-09-10T19:09:47.620834+0000) 2024-09-10T19:10:01.843 INFO:journalctl@ceph.osd.6.smithi149.stdout:Sep 10 19:10:01 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-6[49840]: 2024-09-10T19:10:01.434+0000 7f20c5917640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.47:6807 osd.1 since back 2024-09-10T19:09:33.521625+0000 front 2024-09-10T19:09:33.521700+0000 (oldest deadline 2024-09-10T19:09:59.421548+0000) 2024-09-10T19:10:01.843 INFO:journalctl@ceph.osd.5.smithi149.stdout:Sep 10 19:10:01 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-5[44040]: 2024-09-10T19:10:01.350+0000 7fd4a9b91640 -1 osd.5 67 heartbeat_check: no reply from 172.21.15.47:6803 osd.0 since back 2024-09-10T19:09:22.439498+0000 front 2024-09-10T19:09:22.439454+0000 (oldest deadline 2024-09-10T19:09:50.040068+0000) 2024-09-10T19:10:01.843 INFO:journalctl@ceph.osd.5.smithi149.stdout:Sep 10 19:10:01 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-5[44040]: 2024-09-10T19:10:01.350+0000 7fd4a9b91640 -1 osd.5 67 heartbeat_check: no reply from 172.21.15.47:6807 osd.1 since back 2024-09-10T19:09:32.941381+0000 front 2024-09-10T19:09:32.941232+0000 (oldest deadline 2024-09-10T19:09:57.041381+0000) 2024-09-10T19:10:02.311 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:10:01 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7[55520]: 2024-09-10T19:10:01.986+0000 7f042d57e640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.47:6803 osd.0 since back 2024-09-10T19:09:25.558659+0000 front 2024-09-10T19:09:25.558741+0000 (oldest deadline 2024-09-10T19:09:47.858457+0000) 2024-09-10T19:10:02.311 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:10:01 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7[55520]: 2024-09-10T19:10:01.986+0000 7f042d57e640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.47:6807 osd.1 since back 2024-09-10T19:09:31.859874+0000 front 2024-09-10T19:09:31.859940+0000 (oldest deadline 2024-09-10T19:09:55.359754+0000) 2024-09-10T19:10:02.593 INFO:journalctl@ceph.osd.6.smithi149.stdout:Sep 10 19:10:02 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-6[49840]: 2024-09-10T19:10:02.393+0000 7f20c5917640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.47:6803 osd.0 since back 2024-09-10T19:09:25.320938+0000 front 2024-09-10T19:09:25.320862+0000 (oldest deadline 2024-09-10T19:09:47.620834+0000) 2024-09-10T19:10:02.593 INFO:journalctl@ceph.osd.6.smithi149.stdout:Sep 10 19:10:02 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-6[49840]: 2024-09-10T19:10:02.393+0000 7f20c5917640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.47:6807 osd.1 since back 2024-09-10T19:09:33.521625+0000 front 2024-09-10T19:09:33.521700+0000 (oldest deadline 2024-09-10T19:09:59.421548+0000) 2024-09-10T19:10:02.593 INFO:journalctl@ceph.osd.5.smithi149.stdout:Sep 10 19:10:02 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-5[44040]: 2024-09-10T19:10:02.309+0000 7fd4a9b91640 -1 osd.5 67 heartbeat_check: no reply from 172.21.15.47:6803 osd.0 since back 2024-09-10T19:09:22.439498+0000 front 2024-09-10T19:09:22.439454+0000 (oldest deadline 2024-09-10T19:09:50.040068+0000) 2024-09-10T19:10:02.593 INFO:journalctl@ceph.osd.5.smithi149.stdout:Sep 10 19:10:02 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-5[44040]: 2024-09-10T19:10:02.309+0000 7fd4a9b91640 -1 osd.5 67 heartbeat_check: no reply from 172.21.15.47:6807 osd.1 since back 2024-09-10T19:09:32.941381+0000 front 2024-09-10T19:09:32.941232+0000 (oldest deadline 2024-09-10T19:09:57.041381+0000) 2024-09-10T19:10:03.269 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:10:02 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7[55520]: 2024-09-10T19:10:02.944+0000 7f042d57e640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.47:6803 osd.0 since back 2024-09-10T19:09:25.558659+0000 front 2024-09-10T19:09:25.558741+0000 (oldest deadline 2024-09-10T19:09:47.858457+0000) 2024-09-10T19:10:03.269 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:10:02 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7[55520]: 2024-09-10T19:10:02.944+0000 7f042d57e640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.47:6807 osd.1 since back 2024-09-10T19:09:31.859874+0000 front 2024-09-10T19:09:31.859940+0000 (oldest deadline 2024-09-10T19:09:55.359754+0000) 2024-09-10T19:10:03.593 INFO:journalctl@ceph.osd.6.smithi149.stdout:Sep 10 19:10:03 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-6[49840]: 2024-09-10T19:10:03.365+0000 7f20c5917640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.47:6803 osd.0 since back 2024-09-10T19:09:25.320938+0000 front 2024-09-10T19:09:25.320862+0000 (oldest deadline 2024-09-10T19:09:47.620834+0000) 2024-09-10T19:10:03.593 INFO:journalctl@ceph.osd.6.smithi149.stdout:Sep 10 19:10:03 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-6[49840]: 2024-09-10T19:10:03.365+0000 7f20c5917640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.47:6807 osd.1 since back 2024-09-10T19:09:33.521625+0000 front 2024-09-10T19:09:33.521700+0000 (oldest deadline 2024-09-10T19:09:59.421548+0000) 2024-09-10T19:10:03.593 INFO:journalctl@ceph.osd.5.smithi149.stdout:Sep 10 19:10:03 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-5[44040]: 2024-09-10T19:10:03.267+0000 7fd4a9b91640 -1 osd.5 67 heartbeat_check: no reply from 172.21.15.47:6803 osd.0 since back 2024-09-10T19:09:22.439498+0000 front 2024-09-10T19:09:22.439454+0000 (oldest deadline 2024-09-10T19:09:50.040068+0000) 2024-09-10T19:10:03.593 INFO:journalctl@ceph.osd.5.smithi149.stdout:Sep 10 19:10:03 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-5[44040]: 2024-09-10T19:10:03.267+0000 7fd4a9b91640 -1 osd.5 67 heartbeat_check: no reply from 172.21.15.47:6807 osd.1 since back 2024-09-10T19:09:32.941381+0000 front 2024-09-10T19:09:32.941232+0000 (oldest deadline 2024-09-10T19:09:57.041381+0000) 2024-09-10T19:10:04.305 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:10:03 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7[55520]: 2024-09-10T19:10:03.945+0000 7f042d57e640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.47:6803 osd.0 since back 2024-09-10T19:09:25.558659+0000 front 2024-09-10T19:09:25.558741+0000 (oldest deadline 2024-09-10T19:09:47.858457+0000) 2024-09-10T19:10:04.305 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:10:03 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7[55520]: 2024-09-10T19:10:03.945+0000 7f042d57e640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.47:6807 osd.1 since back 2024-09-10T19:09:31.859874+0000 front 2024-09-10T19:09:31.859940+0000 (oldest deadline 2024-09-10T19:09:55.359754+0000) 2024-09-10T19:10:04.593 INFO:journalctl@ceph.osd.6.smithi149.stdout:Sep 10 19:10:04 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-6[49840]: 2024-09-10T19:10:04.391+0000 7f20c5917640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.47:6803 osd.0 since back 2024-09-10T19:09:25.320938+0000 front 2024-09-10T19:09:25.320862+0000 (oldest deadline 2024-09-10T19:09:47.620834+0000) 2024-09-10T19:10:04.593 INFO:journalctl@ceph.osd.6.smithi149.stdout:Sep 10 19:10:04 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-6[49840]: 2024-09-10T19:10:04.391+0000 7f20c5917640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.47:6807 osd.1 since back 2024-09-10T19:09:33.521625+0000 front 2024-09-10T19:09:33.521700+0000 (oldest deadline 2024-09-10T19:09:59.421548+0000) 2024-09-10T19:10:04.594 INFO:journalctl@ceph.osd.5.smithi149.stdout:Sep 10 19:10:04 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-5[44040]: 2024-09-10T19:10:04.303+0000 7fd4a9b91640 -1 osd.5 67 heartbeat_check: no reply from 172.21.15.47:6803 osd.0 since back 2024-09-10T19:09:22.439498+0000 front 2024-09-10T19:09:22.439454+0000 (oldest deadline 2024-09-10T19:09:50.040068+0000) 2024-09-10T19:10:04.594 INFO:journalctl@ceph.osd.5.smithi149.stdout:Sep 10 19:10:04 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-5[44040]: 2024-09-10T19:10:04.303+0000 7fd4a9b91640 -1 osd.5 67 heartbeat_check: no reply from 172.21.15.47:6807 osd.1 since back 2024-09-10T19:09:32.941381+0000 front 2024-09-10T19:09:32.941232+0000 (oldest deadline 2024-09-10T19:09:57.041381+0000) 2024-09-10T19:10:04.594 INFO:journalctl@ceph.osd.5.smithi149.stdout:Sep 10 19:10:04 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-5[44040]: 2024-09-10T19:10:04.303+0000 7fd4a9b91640 -1 osd.5 67 heartbeat_check: no reply from 172.21.15.47:6811 osd.2 since back 2024-09-10T19:09:41.042670+0000 front 2024-09-10T19:09:41.042753+0000 (oldest deadline 2024-09-10T19:10:03.342609+0000) 2024-09-10T19:10:05.268 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:10:04 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7[55520]: 2024-09-10T19:10:04.972+0000 7f042d57e640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.47:6803 osd.0 since back 2024-09-10T19:09:25.558659+0000 front 2024-09-10T19:09:25.558741+0000 (oldest deadline 2024-09-10T19:09:47.858457+0000) 2024-09-10T19:10:05.268 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:10:04 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7[55520]: 2024-09-10T19:10:04.972+0000 7f042d57e640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.47:6807 osd.1 since back 2024-09-10T19:09:31.859874+0000 front 2024-09-10T19:09:31.859940+0000 (oldest deadline 2024-09-10T19:09:55.359754+0000) 2024-09-10T19:10:05.593 INFO:journalctl@ceph.osd.6.smithi149.stdout:Sep 10 19:10:05 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-6[49840]: 2024-09-10T19:10:05.415+0000 7f20c5917640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.47:6803 osd.0 since back 2024-09-10T19:09:25.320938+0000 front 2024-09-10T19:09:25.320862+0000 (oldest deadline 2024-09-10T19:09:47.620834+0000) 2024-09-10T19:10:05.593 INFO:journalctl@ceph.osd.6.smithi149.stdout:Sep 10 19:10:05 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-6[49840]: 2024-09-10T19:10:05.415+0000 7f20c5917640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.47:6807 osd.1 since back 2024-09-10T19:09:33.521625+0000 front 2024-09-10T19:09:33.521700+0000 (oldest deadline 2024-09-10T19:09:59.421548+0000) 2024-09-10T19:10:05.593 INFO:journalctl@ceph.osd.6.smithi149.stdout:Sep 10 19:10:05 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-6[49840]: 2024-09-10T19:10:05.415+0000 7f20c5917640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.47:6811 osd.2 since back 2024-09-10T19:09:39.422059+0000 front 2024-09-10T19:09:39.422115+0000 (oldest deadline 2024-09-10T19:10:04.722054+0000) 2024-09-10T19:10:05.593 INFO:journalctl@ceph.osd.5.smithi149.stdout:Sep 10 19:10:05 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-5[44040]: 2024-09-10T19:10:05.266+0000 7fd4a9b91640 -1 osd.5 67 heartbeat_check: no reply from 172.21.15.47:6803 osd.0 since back 2024-09-10T19:09:22.439498+0000 front 2024-09-10T19:09:22.439454+0000 (oldest deadline 2024-09-10T19:09:50.040068+0000) 2024-09-10T19:10:05.593 INFO:journalctl@ceph.osd.5.smithi149.stdout:Sep 10 19:10:05 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-5[44040]: 2024-09-10T19:10:05.266+0000 7fd4a9b91640 -1 osd.5 67 heartbeat_check: no reply from 172.21.15.47:6807 osd.1 since back 2024-09-10T19:09:32.941381+0000 front 2024-09-10T19:09:32.941232+0000 (oldest deadline 2024-09-10T19:09:57.041381+0000) 2024-09-10T19:10:05.593 INFO:journalctl@ceph.osd.5.smithi149.stdout:Sep 10 19:10:05 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-5[44040]: 2024-09-10T19:10:05.266+0000 7fd4a9b91640 -1 osd.5 67 heartbeat_check: no reply from 172.21.15.47:6811 osd.2 since back 2024-09-10T19:09:41.042670+0000 front 2024-09-10T19:09:41.042753+0000 (oldest deadline 2024-09-10T19:10:03.342609+0000) 2024-09-10T19:10:06.315 INFO:journalctl@ceph.osd.5.smithi149.stdout:Sep 10 19:10:06 smithi149 podman[71068]: 2024-09-10 19:10:06.049153534 +0000 UTC m=+5.582906361 container died 0ed3d971a000c4cdc54cd0e94ec4114c223369f9228c46f18aab76c8def28da9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T19:10:06.316 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:10:06 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7[55520]: 2024-09-10T19:10:06.009+0000 7f042d57e640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.47:6803 osd.0 since back 2024-09-10T19:09:25.558659+0000 front 2024-09-10T19:09:25.558741+0000 (oldest deadline 2024-09-10T19:09:47.858457+0000) 2024-09-10T19:10:06.316 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:10:06 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7[55520]: 2024-09-10T19:10:06.009+0000 7f042d57e640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.47:6807 osd.1 since back 2024-09-10T19:09:31.859874+0000 front 2024-09-10T19:09:31.859940+0000 (oldest deadline 2024-09-10T19:09:55.359754+0000) 2024-09-10T19:10:06.316 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:10:06 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7[55520]: 2024-09-10T19:10:06.009+0000 7f042d57e640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.47:6811 osd.2 since back 2024-09-10T19:09:41.761199+0000 front 2024-09-10T19:09:41.761146+0000 (oldest deadline 2024-09-10T19:10:05.261164+0000) 2024-09-10T19:10:06.593 INFO:journalctl@ceph.osd.6.smithi149.stdout:Sep 10 19:10:06 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-6[49840]: 2024-09-10T19:10:06.413+0000 7f20c5917640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.47:6803 osd.0 since back 2024-09-10T19:09:25.320938+0000 front 2024-09-10T19:09:25.320862+0000 (oldest deadline 2024-09-10T19:09:47.620834+0000) 2024-09-10T19:10:06.593 INFO:journalctl@ceph.osd.6.smithi149.stdout:Sep 10 19:10:06 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-6[49840]: 2024-09-10T19:10:06.413+0000 7f20c5917640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.47:6807 osd.1 since back 2024-09-10T19:09:33.521625+0000 front 2024-09-10T19:09:33.521700+0000 (oldest deadline 2024-09-10T19:09:59.421548+0000) 2024-09-10T19:10:06.593 INFO:journalctl@ceph.osd.6.smithi149.stdout:Sep 10 19:10:06 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-6[49840]: 2024-09-10T19:10:06.413+0000 7f20c5917640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.47:6811 osd.2 since back 2024-09-10T19:09:39.422059+0000 front 2024-09-10T19:09:39.422115+0000 (oldest deadline 2024-09-10T19:10:04.722054+0000) 2024-09-10T19:10:06.593 INFO:journalctl@ceph.osd.5.smithi149.stdout:Sep 10 19:10:06 smithi149 podman[71068]: 2024-09-10 19:10:06.405685328 +0000 UTC m=+5.939438153 container cleanup 0ed3d971a000c4cdc54cd0e94ec4114c223369f9228c46f18aab76c8def28da9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-5, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T19:10:06.593 INFO:journalctl@ceph.osd.5.smithi149.stdout:Sep 10 19:10:06 smithi149 bash[71068]: ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-5 2024-09-10T19:10:06.990 INFO:journalctl@ceph.osd.5.smithi149.stdout:Sep 10 19:10:06 smithi149 podman[71083]: 2024-09-10 19:10:06.690401309 +0000 UTC m=+0.633724808 container remove 0ed3d971a000c4cdc54cd0e94ec4114c223369f9228c46f18aab76c8def28da9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T19:10:06.990 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:10:06 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7[55520]: 2024-09-10T19:10:06.988+0000 7f042d57e640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.47:6803 osd.0 since back 2024-09-10T19:09:25.558659+0000 front 2024-09-10T19:09:25.558741+0000 (oldest deadline 2024-09-10T19:09:47.858457+0000) 2024-09-10T19:10:06.991 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:10:06 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7[55520]: 2024-09-10T19:10:06.988+0000 7f042d57e640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.47:6807 osd.1 since back 2024-09-10T19:09:31.859874+0000 front 2024-09-10T19:09:31.859940+0000 (oldest deadline 2024-09-10T19:09:55.359754+0000) 2024-09-10T19:10:06.991 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:10:06 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7[55520]: 2024-09-10T19:10:06.988+0000 7f042d57e640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.47:6811 osd.2 since back 2024-09-10T19:09:41.761199+0000 front 2024-09-10T19:09:41.761146+0000 (oldest deadline 2024-09-10T19:10:05.261164+0000) 2024-09-10T19:10:07.262 INFO:journalctl@ceph.osd.5.smithi149.stdout:Sep 10 19:10:07 smithi149 podman[71175]: 2024-09-10 19:10:06.908125957 +0000 UTC m=+0.016227549 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16 2024-09-10T19:10:07.263 INFO:journalctl@ceph.osd.5.smithi149.stdout:Sep 10 19:10:07 smithi149 podman[71175]: 2024-09-10 19:10:07.018424794 +0000 UTC m=+0.126526419 container create 1938b73c7509b84107dd113cd882ad56a66412e6f97e4d119ddef33650f688c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-10T19:10:07.263 INFO:journalctl@ceph.osd.5.smithi149.stdout:Sep 10 19:10:07 smithi149 podman[71175]: 2024-09-10 19:10:07.261396722 +0000 UTC m=+0.369498304 container init 1938b73c7509b84107dd113cd882ad56a66412e6f97e4d119ddef33650f688c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-10T19:10:07.593 INFO:journalctl@ceph.osd.6.smithi149.stdout:Sep 10 19:10:07 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-6[49840]: 2024-09-10T19:10:07.388+0000 7f20c5917640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.47:6803 osd.0 since back 2024-09-10T19:09:25.320938+0000 front 2024-09-10T19:09:25.320862+0000 (oldest deadline 2024-09-10T19:09:47.620834+0000) 2024-09-10T19:10:07.593 INFO:journalctl@ceph.osd.6.smithi149.stdout:Sep 10 19:10:07 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-6[49840]: 2024-09-10T19:10:07.388+0000 7f20c5917640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.47:6807 osd.1 since back 2024-09-10T19:09:33.521625+0000 front 2024-09-10T19:09:33.521700+0000 (oldest deadline 2024-09-10T19:09:59.421548+0000) 2024-09-10T19:10:07.593 INFO:journalctl@ceph.osd.6.smithi149.stdout:Sep 10 19:10:07 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-6[49840]: 2024-09-10T19:10:07.388+0000 7f20c5917640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.47:6811 osd.2 since back 2024-09-10T19:09:39.422059+0000 front 2024-09-10T19:09:39.422115+0000 (oldest deadline 2024-09-10T19:10:04.722054+0000) 2024-09-10T19:10:07.594 INFO:journalctl@ceph.osd.5.smithi149.stdout:Sep 10 19:10:07 smithi149 podman[71175]: 2024-09-10 19:10:07.264801673 +0000 UTC m=+0.372903258 container start 1938b73c7509b84107dd113cd882ad56a66412e6f97e4d119ddef33650f688c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-5-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739) 2024-09-10T19:10:07.594 INFO:journalctl@ceph.osd.5.smithi149.stdout:Sep 10 19:10:07 smithi149 podman[71175]: 2024-09-10 19:10:07.344571299 +0000 UTC m=+0.452672901 container attach 1938b73c7509b84107dd113cd882ad56a66412e6f97e4d119ddef33650f688c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T19:10:07.594 INFO:journalctl@ceph.osd.5.smithi149.stdout:Sep 10 19:10:07 smithi149 podman[71175]: 2024-09-10 19:10:07.441335241 +0000 UTC m=+0.549436847 container died 1938b73c7509b84107dd113cd882ad56a66412e6f97e4d119ddef33650f688c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-10T19:10:07.969 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968@osd.5.service' 2024-09-10T19:10:07.992 INFO:journalctl@ceph.osd.5.smithi149.stdout:Sep 10 19:10:07 smithi149 podman[71200]: 2024-09-10 19:10:07.948062497 +0000 UTC m=+0.500801600 container remove 1938b73c7509b84107dd113cd882ad56a66412e6f97e4d119ddef33650f688c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-10T19:10:07.993 INFO:journalctl@ceph.osd.5.smithi149.stdout:Sep 10 19:10:07 smithi149 systemd[1]: ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968@osd.5.service: Deactivated successfully. 2024-09-10T19:10:07.993 INFO:journalctl@ceph.osd.5.smithi149.stdout:Sep 10 19:10:07 smithi149 systemd[1]: Stopped Ceph osd.5 for d8563d7e-6fa5-11ef-bcea-c7b262605968. 2024-09-10T19:10:07.993 INFO:journalctl@ceph.osd.5.smithi149.stdout:Sep 10 19:10:07 smithi149 systemd[1]: ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968@osd.5.service: Consumed 4.870s CPU time. 2024-09-10T19:10:07.993 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:10:07 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7[55520]: 2024-09-10T19:10:07.977+0000 7f042d57e640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.47:6803 osd.0 since back 2024-09-10T19:09:25.558659+0000 front 2024-09-10T19:09:25.558741+0000 (oldest deadline 2024-09-10T19:09:47.858457+0000) 2024-09-10T19:10:07.993 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:10:07 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7[55520]: 2024-09-10T19:10:07.977+0000 7f042d57e640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.47:6807 osd.1 since back 2024-09-10T19:09:31.859874+0000 front 2024-09-10T19:09:31.859940+0000 (oldest deadline 2024-09-10T19:09:55.359754+0000) 2024-09-10T19:10:07.993 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:10:07 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7[55520]: 2024-09-10T19:10:07.977+0000 7f042d57e640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.47:6811 osd.2 since back 2024-09-10T19:09:41.761199+0000 front 2024-09-10T19:09:41.761146+0000 (oldest deadline 2024-09-10T19:10:05.261164+0000) 2024-09-10T19:10:08.008 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T19:10:08.008 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-10T19:10:08.008 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-10T19:10:08.008 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968@osd.6 2024-09-10T19:10:08.327 INFO:journalctl@ceph.osd.6.smithi149.stdout:Sep 10 19:10:08 smithi149 systemd[1]: Stopping Ceph osd.6 for d8563d7e-6fa5-11ef-bcea-c7b262605968... 2024-09-10T19:10:08.593 INFO:journalctl@ceph.osd.6.smithi149.stdout:Sep 10 19:10:08 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-6[49840]: 2024-09-10T19:10:08.326+0000 7f20c82fd640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T19:10:08.593 INFO:journalctl@ceph.osd.6.smithi149.stdout:Sep 10 19:10:08 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-6[49840]: 2024-09-10T19:10:08.326+0000 7f20c82fd640 -1 osd.6 67 *** Got signal Terminated *** 2024-09-10T19:10:08.593 INFO:journalctl@ceph.osd.6.smithi149.stdout:Sep 10 19:10:08 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-6[49840]: 2024-09-10T19:10:08.326+0000 7f20c82fd640 -1 osd.6 67 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T19:10:08.593 INFO:journalctl@ceph.osd.6.smithi149.stdout:Sep 10 19:10:08 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-6[49840]: 2024-09-10T19:10:08.345+0000 7f20c5917640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.47:6803 osd.0 since back 2024-09-10T19:09:25.320938+0000 front 2024-09-10T19:09:25.320862+0000 (oldest deadline 2024-09-10T19:09:47.620834+0000) 2024-09-10T19:10:08.593 INFO:journalctl@ceph.osd.6.smithi149.stdout:Sep 10 19:10:08 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-6[49840]: 2024-09-10T19:10:08.345+0000 7f20c5917640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.47:6807 osd.1 since back 2024-09-10T19:09:33.521625+0000 front 2024-09-10T19:09:33.521700+0000 (oldest deadline 2024-09-10T19:09:59.421548+0000) 2024-09-10T19:10:08.594 INFO:journalctl@ceph.osd.6.smithi149.stdout:Sep 10 19:10:08 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-6[49840]: 2024-09-10T19:10:08.345+0000 7f20c5917640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.47:6811 osd.2 since back 2024-09-10T19:09:39.422059+0000 front 2024-09-10T19:09:39.422115+0000 (oldest deadline 2024-09-10T19:10:04.722054+0000) 2024-09-10T19:10:09.343 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:10:08 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7[55520]: 2024-09-10T19:10:08.972+0000 7f042d57e640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.47:6803 osd.0 since back 2024-09-10T19:09:25.558659+0000 front 2024-09-10T19:09:25.558741+0000 (oldest deadline 2024-09-10T19:09:47.858457+0000) 2024-09-10T19:10:09.343 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:10:08 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7[55520]: 2024-09-10T19:10:08.972+0000 7f042d57e640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.47:6807 osd.1 since back 2024-09-10T19:09:31.859874+0000 front 2024-09-10T19:09:31.859940+0000 (oldest deadline 2024-09-10T19:09:55.359754+0000) 2024-09-10T19:10:09.343 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:10:08 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7[55520]: 2024-09-10T19:10:08.972+0000 7f042d57e640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.47:6811 osd.2 since back 2024-09-10T19:09:41.761199+0000 front 2024-09-10T19:09:41.761146+0000 (oldest deadline 2024-09-10T19:10:05.261164+0000) 2024-09-10T19:10:09.843 INFO:journalctl@ceph.osd.6.smithi149.stdout:Sep 10 19:10:09 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-6[49840]: 2024-09-10T19:10:09.376+0000 7f20c5917640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.47:6803 osd.0 since back 2024-09-10T19:09:25.320938+0000 front 2024-09-10T19:09:25.320862+0000 (oldest deadline 2024-09-10T19:09:47.620834+0000) 2024-09-10T19:10:09.843 INFO:journalctl@ceph.osd.6.smithi149.stdout:Sep 10 19:10:09 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-6[49840]: 2024-09-10T19:10:09.376+0000 7f20c5917640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.47:6807 osd.1 since back 2024-09-10T19:09:33.521625+0000 front 2024-09-10T19:09:33.521700+0000 (oldest deadline 2024-09-10T19:09:59.421548+0000) 2024-09-10T19:10:09.843 INFO:journalctl@ceph.osd.6.smithi149.stdout:Sep 10 19:10:09 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-6[49840]: 2024-09-10T19:10:09.376+0000 7f20c5917640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.47:6811 osd.2 since back 2024-09-10T19:09:39.422059+0000 front 2024-09-10T19:09:39.422115+0000 (oldest deadline 2024-09-10T19:10:04.722054+0000) 2024-09-10T19:10:10.283 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:10:09 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7[55520]: 2024-09-10T19:10:09.945+0000 7f042d57e640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.47:6803 osd.0 since back 2024-09-10T19:09:25.558659+0000 front 2024-09-10T19:09:25.558741+0000 (oldest deadline 2024-09-10T19:09:47.858457+0000) 2024-09-10T19:10:10.283 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:10:09 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7[55520]: 2024-09-10T19:10:09.945+0000 7f042d57e640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.47:6807 osd.1 since back 2024-09-10T19:09:31.859874+0000 front 2024-09-10T19:09:31.859940+0000 (oldest deadline 2024-09-10T19:09:55.359754+0000) 2024-09-10T19:10:10.284 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:10:09 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7[55520]: 2024-09-10T19:10:09.945+0000 7f042d57e640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.47:6811 osd.2 since back 2024-09-10T19:09:41.761199+0000 front 2024-09-10T19:09:41.761146+0000 (oldest deadline 2024-09-10T19:10:05.261164+0000) 2024-09-10T19:10:10.593 INFO:journalctl@ceph.osd.6.smithi149.stdout:Sep 10 19:10:10 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-6[49840]: 2024-09-10T19:10:10.366+0000 7f20c5917640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.47:6803 osd.0 since back 2024-09-10T19:09:25.320938+0000 front 2024-09-10T19:09:25.320862+0000 (oldest deadline 2024-09-10T19:09:47.620834+0000) 2024-09-10T19:10:10.593 INFO:journalctl@ceph.osd.6.smithi149.stdout:Sep 10 19:10:10 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-6[49840]: 2024-09-10T19:10:10.366+0000 7f20c5917640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.47:6807 osd.1 since back 2024-09-10T19:09:33.521625+0000 front 2024-09-10T19:09:33.521700+0000 (oldest deadline 2024-09-10T19:09:59.421548+0000) 2024-09-10T19:10:10.593 INFO:journalctl@ceph.osd.6.smithi149.stdout:Sep 10 19:10:10 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-6[49840]: 2024-09-10T19:10:10.366+0000 7f20c5917640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.47:6811 osd.2 since back 2024-09-10T19:09:39.422059+0000 front 2024-09-10T19:09:39.422115+0000 (oldest deadline 2024-09-10T19:10:04.722054+0000) 2024-09-10T19:10:11.343 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:10:10 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7[55520]: 2024-09-10T19:10:10.980+0000 7f042d57e640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.47:6803 osd.0 since back 2024-09-10T19:09:25.558659+0000 front 2024-09-10T19:09:25.558741+0000 (oldest deadline 2024-09-10T19:09:47.858457+0000) 2024-09-10T19:10:11.343 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:10:10 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7[55520]: 2024-09-10T19:10:10.980+0000 7f042d57e640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.47:6807 osd.1 since back 2024-09-10T19:09:31.859874+0000 front 2024-09-10T19:09:31.859940+0000 (oldest deadline 2024-09-10T19:09:55.359754+0000) 2024-09-10T19:10:11.343 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:10:10 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7[55520]: 2024-09-10T19:10:10.980+0000 7f042d57e640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.47:6811 osd.2 since back 2024-09-10T19:09:41.761199+0000 front 2024-09-10T19:09:41.761146+0000 (oldest deadline 2024-09-10T19:10:05.261164+0000) 2024-09-10T19:10:11.843 INFO:journalctl@ceph.osd.6.smithi149.stdout:Sep 10 19:10:11 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-6[49840]: 2024-09-10T19:10:11.371+0000 7f20c5917640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.47:6803 osd.0 since back 2024-09-10T19:09:25.320938+0000 front 2024-09-10T19:09:25.320862+0000 (oldest deadline 2024-09-10T19:09:47.620834+0000) 2024-09-10T19:10:11.843 INFO:journalctl@ceph.osd.6.smithi149.stdout:Sep 10 19:10:11 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-6[49840]: 2024-09-10T19:10:11.371+0000 7f20c5917640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.47:6807 osd.1 since back 2024-09-10T19:09:33.521625+0000 front 2024-09-10T19:09:33.521700+0000 (oldest deadline 2024-09-10T19:09:59.421548+0000) 2024-09-10T19:10:11.843 INFO:journalctl@ceph.osd.6.smithi149.stdout:Sep 10 19:10:11 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-6[49840]: 2024-09-10T19:10:11.371+0000 7f20c5917640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.47:6811 osd.2 since back 2024-09-10T19:09:39.422059+0000 front 2024-09-10T19:09:39.422115+0000 (oldest deadline 2024-09-10T19:10:04.722054+0000) 2024-09-10T19:10:11.843 INFO:journalctl@ceph.osd.6.smithi149.stdout:Sep 10 19:10:11 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-6[49840]: 2024-09-10T19:10:11.371+0000 7f20c5917640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.47:6815 osd.3 since back 2024-09-10T19:09:48.222914+0000 front 2024-09-10T19:09:48.222899+0000 (oldest deadline 2024-09-10T19:10:10.522781+0000) 2024-09-10T19:10:12.343 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:10:11 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7[55520]: 2024-09-10T19:10:11.964+0000 7f042d57e640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.47:6803 osd.0 since back 2024-09-10T19:09:25.558659+0000 front 2024-09-10T19:09:25.558741+0000 (oldest deadline 2024-09-10T19:09:47.858457+0000) 2024-09-10T19:10:12.343 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:10:11 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7[55520]: 2024-09-10T19:10:11.964+0000 7f042d57e640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.47:6807 osd.1 since back 2024-09-10T19:09:31.859874+0000 front 2024-09-10T19:09:31.859940+0000 (oldest deadline 2024-09-10T19:09:55.359754+0000) 2024-09-10T19:10:12.343 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:10:11 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7[55520]: 2024-09-10T19:10:11.964+0000 7f042d57e640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.47:6811 osd.2 since back 2024-09-10T19:09:41.761199+0000 front 2024-09-10T19:09:41.761146+0000 (oldest deadline 2024-09-10T19:10:05.261164+0000) 2024-09-10T19:10:12.343 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:10:11 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7[55520]: 2024-09-10T19:10:11.964+0000 7f042d57e640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.47:6815 osd.3 since back 2024-09-10T19:09:48.662402+0000 front 2024-09-10T19:09:48.662344+0000 (oldest deadline 2024-09-10T19:10:11.562191+0000) 2024-09-10T19:10:12.843 INFO:journalctl@ceph.osd.6.smithi149.stdout:Sep 10 19:10:12 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-6[49840]: 2024-09-10T19:10:12.361+0000 7f20c5917640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.47:6803 osd.0 since back 2024-09-10T19:09:25.320938+0000 front 2024-09-10T19:09:25.320862+0000 (oldest deadline 2024-09-10T19:09:47.620834+0000) 2024-09-10T19:10:12.843 INFO:journalctl@ceph.osd.6.smithi149.stdout:Sep 10 19:10:12 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-6[49840]: 2024-09-10T19:10:12.361+0000 7f20c5917640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.47:6807 osd.1 since back 2024-09-10T19:09:33.521625+0000 front 2024-09-10T19:09:33.521700+0000 (oldest deadline 2024-09-10T19:09:59.421548+0000) 2024-09-10T19:10:12.843 INFO:journalctl@ceph.osd.6.smithi149.stdout:Sep 10 19:10:12 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-6[49840]: 2024-09-10T19:10:12.361+0000 7f20c5917640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.47:6811 osd.2 since back 2024-09-10T19:09:39.422059+0000 front 2024-09-10T19:09:39.422115+0000 (oldest deadline 2024-09-10T19:10:04.722054+0000) 2024-09-10T19:10:12.843 INFO:journalctl@ceph.osd.6.smithi149.stdout:Sep 10 19:10:12 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-6[49840]: 2024-09-10T19:10:12.361+0000 7f20c5917640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.47:6815 osd.3 since back 2024-09-10T19:09:48.222914+0000 front 2024-09-10T19:09:48.222899+0000 (oldest deadline 2024-09-10T19:10:10.522781+0000) 2024-09-10T19:10:13.343 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:10:12 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7[55520]: 2024-09-10T19:10:12.993+0000 7f042d57e640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.47:6803 osd.0 since back 2024-09-10T19:09:25.558659+0000 front 2024-09-10T19:09:25.558741+0000 (oldest deadline 2024-09-10T19:09:47.858457+0000) 2024-09-10T19:10:13.343 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:10:12 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7[55520]: 2024-09-10T19:10:12.993+0000 7f042d57e640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.47:6807 osd.1 since back 2024-09-10T19:09:31.859874+0000 front 2024-09-10T19:09:31.859940+0000 (oldest deadline 2024-09-10T19:09:55.359754+0000) 2024-09-10T19:10:13.343 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:10:12 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7[55520]: 2024-09-10T19:10:12.993+0000 7f042d57e640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.47:6811 osd.2 since back 2024-09-10T19:09:41.761199+0000 front 2024-09-10T19:09:41.761146+0000 (oldest deadline 2024-09-10T19:10:05.261164+0000) 2024-09-10T19:10:13.344 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:10:12 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7[55520]: 2024-09-10T19:10:12.993+0000 7f042d57e640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.47:6815 osd.3 since back 2024-09-10T19:09:48.662402+0000 front 2024-09-10T19:09:48.662344+0000 (oldest deadline 2024-09-10T19:10:11.562191+0000) 2024-09-10T19:10:14.044 INFO:journalctl@ceph.osd.6.smithi149.stdout:Sep 10 19:10:13 smithi149 podman[71278]: 2024-09-10 19:10:13.756133965 +0000 UTC m=+5.524276428 container died 123763c4b48bad93b9b8e0a3499593c0c855202f398efb428ef12f6a75ce5be1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-6, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T19:10:14.045 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:10:13 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7[55520]: 2024-09-10T19:10:13.993+0000 7f042d57e640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.47:6803 osd.0 since back 2024-09-10T19:09:25.558659+0000 front 2024-09-10T19:09:25.558741+0000 (oldest deadline 2024-09-10T19:09:47.858457+0000) 2024-09-10T19:10:14.045 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:10:13 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7[55520]: 2024-09-10T19:10:13.993+0000 7f042d57e640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.47:6807 osd.1 since back 2024-09-10T19:09:31.859874+0000 front 2024-09-10T19:09:31.859940+0000 (oldest deadline 2024-09-10T19:09:55.359754+0000) 2024-09-10T19:10:14.045 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:10:13 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7[55520]: 2024-09-10T19:10:13.993+0000 7f042d57e640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.47:6811 osd.2 since back 2024-09-10T19:09:41.761199+0000 front 2024-09-10T19:09:41.761146+0000 (oldest deadline 2024-09-10T19:10:05.261164+0000) 2024-09-10T19:10:14.045 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:10:13 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7[55520]: 2024-09-10T19:10:13.993+0000 7f042d57e640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.47:6815 osd.3 since back 2024-09-10T19:09:48.662402+0000 front 2024-09-10T19:09:48.662344+0000 (oldest deadline 2024-09-10T19:10:11.562191+0000) 2024-09-10T19:10:14.343 INFO:journalctl@ceph.osd.6.smithi149.stdout:Sep 10 19:10:14 smithi149 podman[71278]: 2024-09-10 19:10:14.166716248 +0000 UTC m=+5.934858715 container cleanup 123763c4b48bad93b9b8e0a3499593c0c855202f398efb428ef12f6a75ce5be1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-6, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739) 2024-09-10T19:10:14.343 INFO:journalctl@ceph.osd.6.smithi149.stdout:Sep 10 19:10:14 smithi149 bash[71278]: ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-6 2024-09-10T19:10:14.727 INFO:journalctl@ceph.osd.6.smithi149.stdout:Sep 10 19:10:14 smithi149 podman[71295]: 2024-09-10 19:10:14.419230983 +0000 UTC m=+0.654246385 container remove 123763c4b48bad93b9b8e0a3499593c0c855202f398efb428ef12f6a75ce5be1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-6, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_CLEAN=True, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-10T19:10:14.998 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:10:14 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7[55520]: 2024-09-10T19:10:14.968+0000 7f042d57e640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.47:6803 osd.0 since back 2024-09-10T19:09:25.558659+0000 front 2024-09-10T19:09:25.558741+0000 (oldest deadline 2024-09-10T19:09:47.858457+0000) 2024-09-10T19:10:14.998 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:10:14 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7[55520]: 2024-09-10T19:10:14.968+0000 7f042d57e640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.47:6807 osd.1 since back 2024-09-10T19:09:31.859874+0000 front 2024-09-10T19:09:31.859940+0000 (oldest deadline 2024-09-10T19:09:55.359754+0000) 2024-09-10T19:10:14.998 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:10:14 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7[55520]: 2024-09-10T19:10:14.968+0000 7f042d57e640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.47:6811 osd.2 since back 2024-09-10T19:09:41.761199+0000 front 2024-09-10T19:09:41.761146+0000 (oldest deadline 2024-09-10T19:10:05.261164+0000) 2024-09-10T19:10:14.999 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:10:14 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7[55520]: 2024-09-10T19:10:14.968+0000 7f042d57e640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.47:6815 osd.3 since back 2024-09-10T19:09:48.662402+0000 front 2024-09-10T19:09:48.662344+0000 (oldest deadline 2024-09-10T19:10:11.562191+0000) 2024-09-10T19:10:14.999 INFO:journalctl@ceph.osd.6.smithi149.stdout:Sep 10 19:10:14 smithi149 podman[71386]: 2024-09-10 19:10:14.629636418 +0000 UTC m=+0.016177383 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16 2024-09-10T19:10:14.999 INFO:journalctl@ceph.osd.6.smithi149.stdout:Sep 10 19:10:14 smithi149 podman[71386]: 2024-09-10 19:10:14.728390253 +0000 UTC m=+0.114931236 container create 8d6907ea7c5b6d2e155cc66d5c63091430cb64f3bfaa763c396c7ea4d4e8b903 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-10T19:10:14.999 INFO:journalctl@ceph.osd.6.smithi149.stdout:Sep 10 19:10:14 smithi149 podman[71386]: 2024-09-10 19:10:14.930782319 +0000 UTC m=+0.317323260 container init 8d6907ea7c5b6d2e155cc66d5c63091430cb64f3bfaa763c396c7ea4d4e8b903 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T19:10:14.999 INFO:journalctl@ceph.osd.6.smithi149.stdout:Sep 10 19:10:14 smithi149 podman[71386]: 2024-09-10 19:10:14.934101777 +0000 UTC m=+0.320642719 container start 8d6907ea7c5b6d2e155cc66d5c63091430cb64f3bfaa763c396c7ea4d4e8b903 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-6-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-10T19:10:14.999 INFO:journalctl@ceph.osd.6.smithi149.stdout:Sep 10 19:10:14 smithi149 podman[71386]: 2024-09-10 19:10:14.997449288 +0000 UTC m=+0.383990234 container attach 8d6907ea7c5b6d2e155cc66d5c63091430cb64f3bfaa763c396c7ea4d4e8b903 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-6-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-10T19:10:15.343 INFO:journalctl@ceph.osd.6.smithi149.stdout:Sep 10 19:10:15 smithi149 podman[71386]: 2024-09-10 19:10:15.114170367 +0000 UTC m=+0.500711308 container died 8d6907ea7c5b6d2e155cc66d5c63091430cb64f3bfaa763c396c7ea4d4e8b903 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T19:10:15.723 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968@osd.6.service' 2024-09-10T19:10:15.760 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T19:10:15.761 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-10T19:10:15.761 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-10T19:10:15.761 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968@osd.7 2024-09-10T19:10:16.081 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:10:15 smithi149 systemd[1]: Stopping Ceph osd.7 for d8563d7e-6fa5-11ef-bcea-c7b262605968... 2024-09-10T19:10:16.081 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:10:15 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7[55520]: 2024-09-10T19:10:15.928+0000 7f042d57e640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.47:6803 osd.0 since back 2024-09-10T19:09:25.558659+0000 front 2024-09-10T19:09:25.558741+0000 (oldest deadline 2024-09-10T19:09:47.858457+0000) 2024-09-10T19:10:16.082 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:10:15 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7[55520]: 2024-09-10T19:10:15.928+0000 7f042d57e640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.47:6807 osd.1 since back 2024-09-10T19:09:31.859874+0000 front 2024-09-10T19:09:31.859940+0000 (oldest deadline 2024-09-10T19:09:55.359754+0000) 2024-09-10T19:10:16.082 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:10:15 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7[55520]: 2024-09-10T19:10:15.928+0000 7f042d57e640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.47:6811 osd.2 since back 2024-09-10T19:09:41.761199+0000 front 2024-09-10T19:09:41.761146+0000 (oldest deadline 2024-09-10T19:10:05.261164+0000) 2024-09-10T19:10:16.082 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:10:15 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7[55520]: 2024-09-10T19:10:15.928+0000 7f042d57e640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.47:6815 osd.3 since back 2024-09-10T19:09:48.662402+0000 front 2024-09-10T19:09:48.662344+0000 (oldest deadline 2024-09-10T19:10:11.562191+0000) 2024-09-10T19:10:16.343 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:10:16 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7[55520]: 2024-09-10T19:10:16.080+0000 7f0431f68640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T19:10:16.343 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:10:16 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7[55520]: 2024-09-10T19:10:16.080+0000 7f0431f68640 -1 osd.7 67 *** Got signal Terminated *** 2024-09-10T19:10:16.343 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:10:16 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7[55520]: 2024-09-10T19:10:16.080+0000 7f0431f68640 -1 osd.7 67 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T19:10:17.343 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:10:16 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7[55520]: 2024-09-10T19:10:16.965+0000 7f042d57e640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.47:6803 osd.0 since back 2024-09-10T19:09:25.558659+0000 front 2024-09-10T19:09:25.558741+0000 (oldest deadline 2024-09-10T19:09:47.858457+0000) 2024-09-10T19:10:17.343 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:10:16 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7[55520]: 2024-09-10T19:10:16.965+0000 7f042d57e640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.47:6807 osd.1 since back 2024-09-10T19:09:31.859874+0000 front 2024-09-10T19:09:31.859940+0000 (oldest deadline 2024-09-10T19:09:55.359754+0000) 2024-09-10T19:10:17.343 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:10:16 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7[55520]: 2024-09-10T19:10:16.965+0000 7f042d57e640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.47:6811 osd.2 since back 2024-09-10T19:09:41.761199+0000 front 2024-09-10T19:09:41.761146+0000 (oldest deadline 2024-09-10T19:10:05.261164+0000) 2024-09-10T19:10:17.343 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:10:16 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7[55520]: 2024-09-10T19:10:16.965+0000 7f042d57e640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.47:6815 osd.3 since back 2024-09-10T19:09:48.662402+0000 front 2024-09-10T19:09:48.662344+0000 (oldest deadline 2024-09-10T19:10:11.562191+0000) 2024-09-10T19:10:18.343 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:10:17 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7[55520]: 2024-09-10T19:10:17.972+0000 7f042d57e640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.47:6803 osd.0 since back 2024-09-10T19:09:25.558659+0000 front 2024-09-10T19:09:25.558741+0000 (oldest deadline 2024-09-10T19:09:47.858457+0000) 2024-09-10T19:10:18.343 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:10:17 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7[55520]: 2024-09-10T19:10:17.972+0000 7f042d57e640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.47:6807 osd.1 since back 2024-09-10T19:09:31.859874+0000 front 2024-09-10T19:09:31.859940+0000 (oldest deadline 2024-09-10T19:09:55.359754+0000) 2024-09-10T19:10:18.343 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:10:17 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7[55520]: 2024-09-10T19:10:17.972+0000 7f042d57e640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.47:6811 osd.2 since back 2024-09-10T19:09:41.761199+0000 front 2024-09-10T19:09:41.761146+0000 (oldest deadline 2024-09-10T19:10:05.261164+0000) 2024-09-10T19:10:18.343 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:10:17 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7[55520]: 2024-09-10T19:10:17.972+0000 7f042d57e640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.47:6815 osd.3 since back 2024-09-10T19:09:48.662402+0000 front 2024-09-10T19:09:48.662344+0000 (oldest deadline 2024-09-10T19:10:11.562191+0000) 2024-09-10T19:10:19.343 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:10:19 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7[55520]: 2024-09-10T19:10:19.001+0000 7f042d57e640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.47:6803 osd.0 since back 2024-09-10T19:09:25.558659+0000 front 2024-09-10T19:09:25.558741+0000 (oldest deadline 2024-09-10T19:09:47.858457+0000) 2024-09-10T19:10:19.343 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:10:19 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7[55520]: 2024-09-10T19:10:19.001+0000 7f042d57e640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.47:6807 osd.1 since back 2024-09-10T19:09:31.859874+0000 front 2024-09-10T19:09:31.859940+0000 (oldest deadline 2024-09-10T19:09:55.359754+0000) 2024-09-10T19:10:19.343 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:10:19 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7[55520]: 2024-09-10T19:10:19.001+0000 7f042d57e640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.47:6811 osd.2 since back 2024-09-10T19:09:41.761199+0000 front 2024-09-10T19:09:41.761146+0000 (oldest deadline 2024-09-10T19:10:05.261164+0000) 2024-09-10T19:10:19.344 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:10:19 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7[55520]: 2024-09-10T19:10:19.001+0000 7f042d57e640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.47:6815 osd.3 since back 2024-09-10T19:09:48.662402+0000 front 2024-09-10T19:09:48.662344+0000 (oldest deadline 2024-09-10T19:10:11.562191+0000) 2024-09-10T19:10:20.343 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:10:20 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7[55520]: 2024-09-10T19:10:20.021+0000 7f042d57e640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.47:6803 osd.0 since back 2024-09-10T19:09:25.558659+0000 front 2024-09-10T19:09:25.558741+0000 (oldest deadline 2024-09-10T19:09:47.858457+0000) 2024-09-10T19:10:20.343 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:10:20 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7[55520]: 2024-09-10T19:10:20.021+0000 7f042d57e640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.47:6807 osd.1 since back 2024-09-10T19:09:31.859874+0000 front 2024-09-10T19:09:31.859940+0000 (oldest deadline 2024-09-10T19:09:55.359754+0000) 2024-09-10T19:10:20.343 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:10:20 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7[55520]: 2024-09-10T19:10:20.021+0000 7f042d57e640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.47:6811 osd.2 since back 2024-09-10T19:09:41.761199+0000 front 2024-09-10T19:09:41.761146+0000 (oldest deadline 2024-09-10T19:10:05.261164+0000) 2024-09-10T19:10:20.344 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:10:20 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7[55520]: 2024-09-10T19:10:20.021+0000 7f042d57e640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.47:6815 osd.3 since back 2024-09-10T19:09:48.662402+0000 front 2024-09-10T19:09:48.662344+0000 (oldest deadline 2024-09-10T19:10:11.562191+0000) 2024-09-10T19:10:21.343 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:10:21 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7[55520]: 2024-09-10T19:10:21.008+0000 7f042d57e640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.47:6803 osd.0 since back 2024-09-10T19:09:25.558659+0000 front 2024-09-10T19:09:25.558741+0000 (oldest deadline 2024-09-10T19:09:47.858457+0000) 2024-09-10T19:10:21.343 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:10:21 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7[55520]: 2024-09-10T19:10:21.008+0000 7f042d57e640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.47:6807 osd.1 since back 2024-09-10T19:09:31.859874+0000 front 2024-09-10T19:09:31.859940+0000 (oldest deadline 2024-09-10T19:09:55.359754+0000) 2024-09-10T19:10:21.343 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:10:21 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7[55520]: 2024-09-10T19:10:21.008+0000 7f042d57e640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.47:6811 osd.2 since back 2024-09-10T19:09:41.761199+0000 front 2024-09-10T19:09:41.761146+0000 (oldest deadline 2024-09-10T19:10:05.261164+0000) 2024-09-10T19:10:21.343 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:10:21 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7[55520]: 2024-09-10T19:10:21.008+0000 7f042d57e640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.47:6815 osd.3 since back 2024-09-10T19:09:48.662402+0000 front 2024-09-10T19:09:48.662344+0000 (oldest deadline 2024-09-10T19:10:11.562191+0000) 2024-09-10T19:10:21.843 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:10:21 smithi149 podman[71488]: 2024-09-10 19:10:21.569700967 +0000 UTC m=+5.585899913 container died 1ce2605cbb35266a054783f13bbd292934bca8306241973031a3ceba6b74281f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7, ceph=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-10T19:10:22.174 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:10:21 smithi149 podman[71488]: 2024-09-10 19:10:21.946740481 +0000 UTC m=+5.962939423 container cleanup 1ce2605cbb35266a054783f13bbd292934bca8306241973031a3ceba6b74281f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7, io.buildah.version=1.37.2, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T19:10:22.174 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:10:21 smithi149 bash[71488]: ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7 2024-09-10T19:10:22.484 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:10:22 smithi149 podman[71514]: 2024-09-10 19:10:22.173057773 +0000 UTC m=+0.594033709 container remove 1ce2605cbb35266a054783f13bbd292934bca8306241973031a3ceba6b74281f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, ceph=True) 2024-09-10T19:10:22.794 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:10:22 smithi149 podman[71604]: 2024-09-10 19:10:22.386227813 +0000 UTC m=+0.016133001 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16 2024-09-10T19:10:22.794 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:10:22 smithi149 podman[71604]: 2024-09-10 19:10:22.490651302 +0000 UTC m=+0.120556490 container create b9860fcdcb110e36ba876be0c648f0497533cd2995935fb21d36daf1c479ab5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-10T19:10:22.794 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:10:22 smithi149 podman[71604]: 2024-09-10 19:10:22.718514134 +0000 UTC m=+0.348419322 container init b9860fcdcb110e36ba876be0c648f0497533cd2995935fb21d36daf1c479ab5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7-deactivate, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-10T19:10:22.794 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:10:22 smithi149 podman[71604]: 2024-09-10 19:10:22.721766791 +0000 UTC m=+0.351671980 container start b9860fcdcb110e36ba876be0c648f0497533cd2995935fb21d36daf1c479ab5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T19:10:22.794 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:10:22 smithi149 podman[71604]: 2024-09-10 19:10:22.793295606 +0000 UTC m=+0.423200805 container attach b9860fcdcb110e36ba876be0c648f0497533cd2995935fb21d36daf1c479ab5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T19:10:23.093 INFO:journalctl@ceph.osd.7.smithi149.stdout:Sep 10 19:10:22 smithi149 podman[71604]: 2024-09-10 19:10:22.897217589 +0000 UTC m=+0.527122786 container died b9860fcdcb110e36ba876be0c648f0497533cd2995935fb21d36daf1c479ab5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-osd-7-deactivate, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-10T19:10:23.525 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968@osd.7.service' 2024-09-10T19:10:23.565 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T19:10:23.565 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-10T19:10:23.565 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopping rgw.foo.a... 2024-09-10T19:10:23.565 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968@rgw.foo.a 2024-09-10T19:10:23.872 INFO:journalctl@ceph.rgw.foo.a.smithi047.stdout:Sep 10 19:10:23 smithi047 systemd[1]: Stopping Ceph rgw.foo.a for d8563d7e-6fa5-11ef-bcea-c7b262605968... 2024-09-10T19:10:24.289 INFO:journalctl@ceph.rgw.foo.a.smithi047.stdout:Sep 10 19:10:23 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-rgw-foo-a[65552]: 2024-09-10T19:10:23.867+0000 7fe4a5622640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/radosgw -n client.rgw.foo.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T19:10:24.289 INFO:journalctl@ceph.rgw.foo.a.smithi047.stdout:Sep 10 19:10:23 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-rgw-foo-a[65552]: 2024-09-10T19:10:23.868+0000 7fe4a9b2a7c0 -1 shutting down 2024-09-10T19:10:34.162 INFO:journalctl@ceph.rgw.foo.a.smithi047.stdout:Sep 10 19:10:33 smithi047 bash[83121]: time="2024-09-10T19:10:33Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-rgw-foo-a in 10 seconds, resorting to SIGKILL" 2024-09-10T19:10:34.162 INFO:journalctl@ceph.rgw.foo.a.smithi047.stdout:Sep 10 19:10:33 smithi047 podman[83121]: 2024-09-10 19:10:33.886592459 +0000 UTC m=+10.113593305 container died ae5fc6a28e182e68eabc39234247f98905adb5a7460d6b769481def95c00b3b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-rgw-foo-a, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-10T19:10:34.539 INFO:journalctl@ceph.rgw.foo.a.smithi047.stdout:Sep 10 19:10:34 smithi047 podman[83121]: 2024-09-10 19:10:34.375743107 +0000 UTC m=+10.602743948 container cleanup ae5fc6a28e182e68eabc39234247f98905adb5a7460d6b769481def95c00b3b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-rgw-foo-a, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-10T19:10:34.539 INFO:journalctl@ceph.rgw.foo.a.smithi047.stdout:Sep 10 19:10:34 smithi047 bash[83121]: ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-rgw-foo-a 2024-09-10T19:10:34.769 DEBUG:teuthology.orchestra.run.smithi047:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968@rgw.foo.a.service' 2024-09-10T19:10:34.807 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T19:10:34.807 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopped rgw.foo.a 2024-09-10T19:10:34.807 INFO:tasks.cephadm.prometheus.a:Stopping prometheus.a... 2024-09-10T19:10:34.807 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968@prometheus.a 2024-09-10T19:10:35.093 INFO:journalctl@ceph.prometheus.a.smithi149.stdout:Sep 10 19:10:34 smithi149 systemd[1]: Stopping Ceph prometheus.a for d8563d7e-6fa5-11ef-bcea-c7b262605968... 2024-09-10T19:10:35.393 INFO:journalctl@ceph.prometheus.a.smithi149.stdout:Sep 10 19:10:35 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-prometheus-a[65365]: ts=2024-09-10T19:10:35.118Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-09-10T19:10:35.393 INFO:journalctl@ceph.prometheus.a.smithi149.stdout:Sep 10 19:10:35 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-prometheus-a[65365]: ts=2024-09-10T19:10:35.118Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-09-10T19:10:35.393 INFO:journalctl@ceph.prometheus.a.smithi149.stdout:Sep 10 19:10:35 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-prometheus-a[65365]: ts=2024-09-10T19:10:35.118Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-09-10T19:10:35.393 INFO:journalctl@ceph.prometheus.a.smithi149.stdout:Sep 10 19:10:35 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-prometheus-a[65365]: ts=2024-09-10T19:10:35.118Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-09-10T19:10:35.393 INFO:journalctl@ceph.prometheus.a.smithi149.stdout:Sep 10 19:10:35 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-prometheus-a[65365]: ts=2024-09-10T19:10:35.118Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-09-10T19:10:35.393 INFO:journalctl@ceph.prometheus.a.smithi149.stdout:Sep 10 19:10:35 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-prometheus-a[65365]: ts=2024-09-10T19:10:35.118Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-09-10T19:10:35.393 INFO:journalctl@ceph.prometheus.a.smithi149.stdout:Sep 10 19:10:35 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-prometheus-a[65365]: ts=2024-09-10T19:10:35.118Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-09-10T19:10:35.394 INFO:journalctl@ceph.prometheus.a.smithi149.stdout:Sep 10 19:10:35 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-prometheus-a[65365]: ts=2024-09-10T19:10:35.118Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-09-10T19:10:35.394 INFO:journalctl@ceph.prometheus.a.smithi149.stdout:Sep 10 19:10:35 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-prometheus-a[65365]: ts=2024-09-10T19:10:35.118Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-09-10T19:10:35.394 INFO:journalctl@ceph.prometheus.a.smithi149.stdout:Sep 10 19:10:35 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-prometheus-a[65365]: ts=2024-09-10T19:10:35.118Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-09-10T19:10:35.394 INFO:journalctl@ceph.prometheus.a.smithi149.stdout:Sep 10 19:10:35 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-prometheus-a[65365]: ts=2024-09-10T19:10:35.118Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-09-10T19:10:35.394 INFO:journalctl@ceph.prometheus.a.smithi149.stdout:Sep 10 19:10:35 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-prometheus-a[65365]: ts=2024-09-10T19:10:35.119Z caller=main.go:1147 level=info msg="See you next time!" 2024-09-10T19:10:35.394 INFO:journalctl@ceph.prometheus.a.smithi149.stdout:Sep 10 19:10:35 smithi149 podman[71707]: 2024-09-10 19:10:35.128553806 +0000 UTC m=+0.124333244 container died d54b0a8cd0949c243715347ede78bc2eefe91d84159da84c9e535101d433df50 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-10T19:10:35.743 INFO:journalctl@ceph.prometheus.a.smithi149.stdout:Sep 10 19:10:35 smithi149 podman[71707]: 2024-09-10 19:10:35.499522076 +0000 UTC m=+0.495301511 container cleanup d54b0a8cd0949c243715347ede78bc2eefe91d84159da84c9e535101d433df50 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-10T19:10:35.743 INFO:journalctl@ceph.prometheus.a.smithi149.stdout:Sep 10 19:10:35 smithi149 bash[71707]: ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-prometheus-a 2024-09-10T19:10:35.876 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968@prometheus.a.service' 2024-09-10T19:10:35.911 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T19:10:35.911 INFO:tasks.cephadm.prometheus.a:Stopped prometheus.a 2024-09-10T19:10:35.911 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d8563d7e-6fa5-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-10T19:10:37.911 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Sep 10 19:10:37 smithi047 systemd[1]: Stopping Ceph alertmanager.a for d8563d7e-6fa5-11ef-bcea-c7b262605968... 2024-09-10T19:10:37.911 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Sep 10 19:10:37 smithi047 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-alertmanager-a[74096]: ts=2024-09-10T19:10:37.889Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-09-10T19:10:38.239 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Sep 10 19:10:37 smithi047 podman[83579]: 2024-09-10 19:10:37.910424829 +0000 UTC m=+0.123239108 container died 197b90c742dbe35cf70ec0f3bba1dd69430403fc972014b6a9b4b9e6895e2808 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-10T19:10:38.239 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Sep 10 19:10:38 smithi047 podman[83579]: 2024-09-10 19:10:38.23901904 +0000 UTC m=+0.451833307 container cleanup 197b90c742dbe35cf70ec0f3bba1dd69430403fc972014b6a9b4b9e6895e2808 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-10T19:10:38.539 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Sep 10 19:10:38 smithi047 bash[83579]: ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-alertmanager-a 2024-09-10T19:10:38.539 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Sep 10 19:10:38 smithi047 podman[83592]: 2024-09-10 19:10:38.398468276 +0000 UTC m=+0.484219188 container remove 197b90c742dbe35cf70ec0f3bba1dd69430403fc972014b6a9b4b9e6895e2808 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-10T19:10:38.540 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Sep 10 19:10:38 smithi047 podman[83592]: 2024-09-10 19:10:38.457384027 +0000 UTC m=+0.543134894 volume remove 3c70c8373c8b39b09d56ec093a63b3acb2a1de00fc23932fec2bc3e9dc6bc2d8 2024-09-10T19:10:39.040 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 10 19:10:38 smithi047 systemd[1]: Stopping Ceph node-exporter.a for d8563d7e-6fa5-11ef-bcea-c7b262605968... 2024-09-10T19:10:39.040 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Sep 10 19:10:38 smithi047 systemd[1]: ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968@alertmanager.a.service: Deactivated successfully. 2024-09-10T19:10:39.040 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Sep 10 19:10:38 smithi047 systemd[1]: Stopped Ceph alertmanager.a for d8563d7e-6fa5-11ef-bcea-c7b262605968. 2024-09-10T19:10:39.040 INFO:journalctl@ceph.alertmanager.a.smithi047.stdout:Sep 10 19:10:38 smithi047 systemd[1]: ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968@alertmanager.a.service: Consumed 1.156s CPU time. 2024-09-10T19:10:39.420 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 10 19:10:39 smithi047 podman[83700]: 2024-09-10 19:10:39.154137592 +0000 UTC m=+0.111950426 container died 39204f3adcb67a79738d76e4a7e7a99d78efce4167a277308478638a295ab599 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-10T19:10:39.769 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 10 19:10:39 smithi047 podman[83700]: 2024-09-10 19:10:39.517919123 +0000 UTC m=+0.475731956 container cleanup 39204f3adcb67a79738d76e4a7e7a99d78efce4167a277308478638a295ab599 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-10T19:10:39.769 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 10 19:10:39 smithi047 bash[83700]: ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-a 2024-09-10T19:10:40.039 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 10 19:10:39 smithi047 podman[83712]: 2024-09-10 19:10:39.768923021 +0000 UTC m=+0.618670464 container remove 39204f3adcb67a79738d76e4a7e7a99d78efce4167a277308478638a295ab599 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-10T19:10:40.040 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 10 19:10:39 smithi047 systemd[1]: ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968@node-exporter.a.service: Main process exited, code=exited, status=143/n/a 2024-09-10T19:10:40.040 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 10 19:10:39 smithi047 systemd[1]: ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968@node-exporter.a.service: Failed with result 'exit-code'. 2024-09-10T19:10:40.040 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 10 19:10:39 smithi047 systemd[1]: Stopped Ceph node-exporter.a for d8563d7e-6fa5-11ef-bcea-c7b262605968. 2024-09-10T19:10:40.040 INFO:journalctl@ceph.node-exporter.a.smithi047.stdout:Sep 10 19:10:39 smithi047 systemd[1]: ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968@node-exporter.a.service: Consumed 2.571s CPU time. 2024-09-10T19:10:40.579 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d8563d7e-6fa5-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-10T19:10:42.093 INFO:journalctl@ceph.iscsi.iscsi.a.smithi149.stdout:Sep 10 19:10:41 smithi149 systemd[1]: Stopping Ceph iscsi.iscsi.a for d8563d7e-6fa5-11ef-bcea-c7b262605968... 2024-09-10T19:10:42.593 INFO:journalctl@ceph.iscsi.iscsi.a.smithi149.stdout:Sep 10 19:10:42 smithi149 ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-iscsi-iscsi-a[60538]: debug Shutdown received 2024-09-10T19:10:52.456 INFO:journalctl@ceph.iscsi.iscsi.a.smithi149.stdout:Sep 10 19:10:52 smithi149 bash[72084]: time="2024-09-10T19:10:52Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-iscsi-iscsi-a in 10 seconds, resorting to SIGKILL" 2024-09-10T19:10:52.457 INFO:journalctl@ceph.iscsi.iscsi.a.smithi149.stdout:Sep 10 19:10:52 smithi149 podman[72084]: 2024-09-10 19:10:52.17864305 +0000 UTC m=+10.129317245 container died 1167f4d9967915690ec2801e3a750df65b81609f79f10f7046fb5ddfd5e40185 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-iscsi-iscsi-a, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True) 2024-09-10T19:10:52.813 INFO:journalctl@ceph.iscsi.iscsi.a.smithi149.stdout:Sep 10 19:10:52 smithi149 podman[72084]: 2024-09-10 19:10:52.578193673 +0000 UTC m=+10.528867867 container cleanup 1167f4d9967915690ec2801e3a750df65b81609f79f10f7046fb5ddfd5e40185 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-iscsi-iscsi-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-10T19:10:52.813 INFO:journalctl@ceph.iscsi.iscsi.a.smithi149.stdout:Sep 10 19:10:52 smithi149 bash[72084]: ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-iscsi-iscsi-a 2024-09-10T19:10:53.093 INFO:journalctl@ceph.iscsi.iscsi.a.smithi149.stdout:Sep 10 19:10:52 smithi149 podman[72097]: 2024-09-10 19:10:52.812534206 +0000 UTC m=+0.634107614 container remove 1167f4d9967915690ec2801e3a750df65b81609f79f10f7046fb5ddfd5e40185 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-iscsi-iscsi-a, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, CEPH_POINT_RELEASE=) 2024-09-10T19:10:53.093 INFO:journalctl@ceph.iscsi.iscsi.a.smithi149.stdout:Sep 10 19:10:52 smithi149 systemd[1]: ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968@iscsi.iscsi.a.service: Main process exited, code=exited, status=137/n/a 2024-09-10T19:10:53.093 INFO:journalctl@ceph.iscsi.iscsi.a.smithi149.stdout:Sep 10 19:10:53 smithi149 podman[72143]: 2024-09-10 19:10:53.059156006 +0000 UTC m=+0.113514831 container died b15be0b684a6f508edeb120ce216dbfad0c9d4f70f7615e8b4fb52e1a6624574 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T19:10:53.543 INFO:journalctl@ceph.iscsi.iscsi.a.smithi149.stdout:Sep 10 19:10:53 smithi149 podman[72143]: 2024-09-10 19:10:53.332403829 +0000 UTC m=+0.386762653 container cleanup b15be0b684a6f508edeb120ce216dbfad0c9d4f70f7615e8b4fb52e1a6624574 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T19:10:53.543 INFO:journalctl@ceph.iscsi.iscsi.a.smithi149.stdout:Sep 10 19:10:53 smithi149 bash[72143]: ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu 2024-09-10T19:10:53.543 INFO:journalctl@ceph.iscsi.iscsi.a.smithi149.stdout:Sep 10 19:10:53 smithi149 systemd[1]: ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968@iscsi.iscsi.a.service: Failed with result 'exit-code'. 2024-09-10T19:10:53.543 INFO:journalctl@ceph.iscsi.iscsi.a.smithi149.stdout:Sep 10 19:10:53 smithi149 systemd[1]: ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968@iscsi.iscsi.a.service: Unit process 60385 (conmon) remains running after unit stopped. 2024-09-10T19:10:53.543 INFO:journalctl@ceph.iscsi.iscsi.a.smithi149.stdout:Sep 10 19:10:53 smithi149 systemd[1]: ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968@iscsi.iscsi.a.service: Unit process 72155 (podman) remains running after unit stopped. 2024-09-10T19:10:53.543 INFO:journalctl@ceph.iscsi.iscsi.a.smithi149.stdout:Sep 10 19:10:53 smithi149 systemd[1]: Stopped Ceph iscsi.iscsi.a for d8563d7e-6fa5-11ef-bcea-c7b262605968. 2024-09-10T19:10:53.543 INFO:journalctl@ceph.iscsi.iscsi.a.smithi149.stdout:Sep 10 19:10:53 smithi149 systemd[1]: ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968@iscsi.iscsi.a.service: Consumed 2.643s CPU time. 2024-09-10T19:10:53.843 INFO:journalctl@ceph.iscsi.iscsi.a.smithi149.stdout:Sep 10 19:10:53 smithi149 podman[72155]: 2024-09-10 19:10:53.541929616 +0000 UTC m=+0.482553379 container remove b15be0b684a6f508edeb120ce216dbfad0c9d4f70f7615e8b4fb52e1a6624574 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.build-date=20240731) 2024-09-10T19:10:53.843 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:10:53 smithi149 systemd[1]: Stopping Ceph grafana.a for d8563d7e-6fa5-11ef-bcea-c7b262605968... 2024-09-10T19:10:54.283 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:10:53 smithi149 podman[72273]: 2024-09-10 19:10:53.973536237 +0000 UTC m=+0.099620019 container died 53b5807bf7ce1a9bfe8ac992be0754db06c7d6875c3372f0cfe6a028746d06e7 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, vcs-type=git, vendor=Red Hat, Inc., architecture=x86_64, io.k8s.display-name=Red Hat Universal Base Image 8, io.openshift.tags=base rhel8, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., io.openshift.expose-services=, description=Ceph Grafana Container, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, version=8.5, build-date=2022-04-29T11:48:11.055897, io.buildah.version=1.32.0, com.redhat.component=ubi8-container, distribution-scope=public, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, release=239.1651231664, name=ubi8, summary=Grafana Container configured for Ceph mgr/dashboard integration, maintainer=Paul Cuzner ) 2024-09-10T19:10:54.538 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:10:54 smithi149 podman[72273]: 2024-09-10 19:10:54.42957896 +0000 UTC m=+0.555662731 container cleanup 53b5807bf7ce1a9bfe8ac992be0754db06c7d6875c3372f0cfe6a028746d06e7 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a, io.buildah.version=1.32.0, io.openshift.tags=base rhel8, release=239.1651231664, architecture=x86_64, maintainer=Paul Cuzner , version=8.5, build-date=2022-04-29T11:48:11.055897, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, com.redhat.component=ubi8-container, summary=Grafana Container configured for Ceph mgr/dashboard integration, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, vcs-type=git, vendor=Red Hat, Inc., io.openshift.expose-services=, name=ubi8, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., io.k8s.display-name=Red Hat Universal Base Image 8, description=Ceph Grafana Container, distribution-scope=public, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI) 2024-09-10T19:10:54.538 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:10:54 smithi149 bash[72273]: ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a 2024-09-10T19:10:54.843 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:10:54 smithi149 podman[72284]: 2024-09-10 19:10:54.730989081 +0000 UTC m=+0.759715131 container remove 53b5807bf7ce1a9bfe8ac992be0754db06c7d6875c3372f0cfe6a028746d06e7 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-grafana-a, build-date=2022-04-29T11:48:11.055897, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, vcs-type=git, io.k8s.display-name=Red Hat Universal Base Image 8, vendor=Red Hat, Inc., com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, maintainer=Paul Cuzner , url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, version=8.5, io.buildah.version=1.32.0, summary=Grafana Container configured for Ceph mgr/dashboard integration, com.redhat.component=ubi8-container, description=Ceph Grafana Container, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., distribution-scope=public, architecture=x86_64, name=ubi8, io.openshift.tags=base rhel8, release=239.1651231664, io.openshift.expose-services=) 2024-09-10T19:10:54.843 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:10:54 smithi149 systemd[1]: ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968@grafana.a.service: Main process exited, code=exited, status=143/n/a 2024-09-10T19:10:55.343 INFO:journalctl@ceph.node-exporter.b.smithi149.stdout:Sep 10 19:10:55 smithi149 systemd[1]: Stopping Ceph node-exporter.b for d8563d7e-6fa5-11ef-bcea-c7b262605968... 2024-09-10T19:10:55.343 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:10:54 smithi149 systemd[1]: ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968@grafana.a.service: Failed with result 'exit-code'. 2024-09-10T19:10:55.343 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:10:54 smithi149 systemd[1]: Stopped Ceph grafana.a for d8563d7e-6fa5-11ef-bcea-c7b262605968. 2024-09-10T19:10:55.344 INFO:journalctl@ceph.grafana.a.smithi149.stdout:Sep 10 19:10:54 smithi149 systemd[1]: ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968@grafana.a.service: Consumed 6.816s CPU time. 2024-09-10T19:10:55.661 INFO:journalctl@ceph.node-exporter.b.smithi149.stdout:Sep 10 19:10:55 smithi149 podman[72402]: 2024-09-10 19:10:55.353950512 +0000 UTC m=+0.104795211 container died 937888306c31cacb2e464ed48db00c47d62b74493c13890a6176a254ed0c0694 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-10T19:10:55.661 INFO:journalctl@ceph.node-exporter.b.smithi149.stdout:Sep 10 19:10:55 smithi149 podman[72402]: 2024-09-10 19:10:55.660509076 +0000 UTC m=+0.411353777 container cleanup 937888306c31cacb2e464ed48db00c47d62b74493c13890a6176a254ed0c0694 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-10T19:10:56.072 INFO:journalctl@ceph.node-exporter.b.smithi149.stdout:Sep 10 19:10:55 smithi149 bash[72402]: ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-b 2024-09-10T19:10:56.073 INFO:journalctl@ceph.node-exporter.b.smithi149.stdout:Sep 10 19:10:55 smithi149 podman[72414]: 2024-09-10 19:10:55.86973598 +0000 UTC m=+0.519700155 container remove 937888306c31cacb2e464ed48db00c47d62b74493c13890a6176a254ed0c0694 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-10T19:10:56.073 INFO:journalctl@ceph.node-exporter.b.smithi149.stdout:Sep 10 19:10:55 smithi149 systemd[1]: ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968@node-exporter.b.service: Main process exited, code=exited, status=143/n/a 2024-09-10T19:10:56.343 INFO:journalctl@ceph.node-exporter.b.smithi149.stdout:Sep 10 19:10:56 smithi149 systemd[1]: ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968@node-exporter.b.service: Failed with result 'exit-code'. 2024-09-10T19:10:56.343 INFO:journalctl@ceph.node-exporter.b.smithi149.stdout:Sep 10 19:10:56 smithi149 systemd[1]: Stopped Ceph node-exporter.b for d8563d7e-6fa5-11ef-bcea-c7b262605968. 2024-09-10T19:10:56.343 INFO:journalctl@ceph.node-exporter.b.smithi149.stdout:Sep 10 19:10:56 smithi149 systemd[1]: ceph-d8563d7e-6fa5-11ef-bcea-c7b262605968@node-exporter.b.service: Consumed 2.450s CPU time. 2024-09-10T19:10:56.721 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T19:10:56.756 DEBUG:teuthology.orchestra.run.smithi149:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T19:10:56.783 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-10T19:10:56.785 DEBUG:teuthology.misc:Transferring archived files from smithi047:/var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/gabrioux-2024-09-10_17:53:37-orch:cephadm-wip-guits-quincy-2024-09-09-1810-quincy-distro-default-smithi/7900270/remote/smithi047/crash 2024-09-10T19:10:56.785 DEBUG:teuthology.orchestra.run.smithi047:> sudo tar c -f - -C /var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/crash -- . 2024-09-10T19:10:56.842 INFO:teuthology.orchestra.run.smithi047.stderr:tar: /var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-10T19:10:56.843 INFO:teuthology.orchestra.run.smithi047.stderr:tar: Error is not recoverable: exiting now 2024-09-10T19:10:56.845 DEBUG:teuthology.misc:Transferring archived files from smithi149:/var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/gabrioux-2024-09-10_17:53:37-orch:cephadm-wip-guits-quincy-2024-09-09-1810-quincy-distro-default-smithi/7900270/remote/smithi149/crash 2024-09-10T19:10:56.845 DEBUG:teuthology.orchestra.run.smithi149:> sudo tar c -f - -C /var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/crash -- . 2024-09-10T19:10:56.884 INFO:teuthology.orchestra.run.smithi149.stderr:tar: /var/lib/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-10T19:10:56.884 INFO:teuthology.orchestra.run.smithi149.stderr:tar: Error is not recoverable: exiting now 2024-09-10T19:10:56.885 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-10T19:10:56.885 DEBUG:teuthology.orchestra.run.smithi047:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | head -n 1 2024-09-10T19:10:56.915 INFO:teuthology.orchestra.run.smithi047.stderr:grep: /var/log/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/ceph.log: No such file or directory 2024-09-10T19:10:56.917 INFO:tasks.cephadm:Compressing logs... 2024-09-10T19:10:56.917 DEBUG:teuthology.orchestra.run.smithi047:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-10T19:10:56.960 DEBUG:teuthology.orchestra.run.smithi149:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-10T19:10:56.985 INFO:teuthology.orchestra.run.smithi149.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/tcmu-runner.log 2024-09-10T19:10:56.986 INFO:teuthology.orchestra.run.smithi149.stderr:: No such file or directory 2024-09-10T19:10:56.986 INFO:teuthology.orchestra.run.smithi149.stderr:gzip -5 --verbose -- /var/log/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/ceph-osd.7.log 2024-09-10T19:10:56.986 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/tcmu-runner.log: gzip -5 --verbose -- /var/log/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-10T19:10:56.986 INFO:teuthology.orchestra.run.smithi149.stderr: 62.7% -- replaced with /var/log/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/tcmu-runner.log.gz 2024-09-10T19:10:56.986 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/ceph-osd.4.log 2024-09-10T19:10:56.986 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/ceph-osd.6.log 2024-09-10T19:10:56.986 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/ceph-osd.5.log 2024-09-10T19:10:56.990 INFO:teuthology.orchestra.run.smithi047.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/ceph-osd.0.log 2024-09-10T19:10:56.990 INFO:teuthology.orchestra.run.smithi047.stderr:: No such file or directory 2024-09-10T19:10:56.990 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/ceph-osd.1.log 2024-09-10T19:10:56.990 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-10T19:10:56.990 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/ceph-osd.0.log: /var/log/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/ceph-mon.c.log 2024-09-10T19:10:56.990 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/ceph-mon.a.log 2024-09-10T19:10:56.990 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/ceph-osd.2.log 2024-09-10T19:10:56.990 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/ceph-mon.c.log: /var/log/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/ceph-mgr.y.log 2024-09-10T19:10:56.990 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/ceph-client.rgw.foo.a.log 2024-09-10T19:10:56.991 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/ceph-mgr.x.log 2024-09-10T19:10:56.991 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/ceph-mon.b.log 2024-09-10T19:10:56.991 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-10T19:10:56.991 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/ceph-mon.b.log: /var/log/ceph/cephadm.log: 92.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-10T19:10:56.992 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/ceph-osd.3.log 2024-09-10T19:10:56.992 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/ceph-client.rgw.foo.a.log: 58.5% -- replaced with /var/log/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/ceph-client.rgw.foo.a.log.gz 2024-09-10T19:10:56.992 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-10T19:10:56.996 INFO:teuthology.orchestra.run.smithi149.stderr: 91.2% -- replaced with /var/log/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-10T19:10:56.999 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/ceph-osd.3.log: 91.2% -- replaced with /var/log/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-10T19:10:57.003 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/cephadm.log: 91.7% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-10T19:10:57.013 INFO:teuthology.orchestra.run.smithi149.stderr: 88.8% -- replaced with /var/log/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/ceph-mgr.x.log.gz 2024-09-10T19:10:57.147 INFO:teuthology.orchestra.run.smithi149.stderr: 92.2% -- replaced with /var/log/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/ceph-mon.b.log.gz 2024-09-10T19:10:57.205 INFO:teuthology.orchestra.run.smithi047.stderr: 88.8% -- replaced with /var/log/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/ceph-mgr.y.log.gz 2024-09-10T19:10:57.210 INFO:teuthology.orchestra.run.smithi047.stderr: 92.2% -- replaced with /var/log/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/ceph-mon.c.log.gz 2024-09-10T19:10:57.279 INFO:teuthology.orchestra.run.smithi149.stderr: 92.9% -- replaced with /var/log/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/ceph-osd.6.log.gz 2024-09-10T19:10:57.331 INFO:teuthology.orchestra.run.smithi149.stderr: 93.2% -- replaced with /var/log/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/ceph-osd.7.log.gz 2024-09-10T19:10:57.373 INFO:teuthology.orchestra.run.smithi149.stderr: 93.0% -- replaced with /var/log/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/ceph-osd.5.log.gz 2024-09-10T19:10:57.388 INFO:teuthology.orchestra.run.smithi047.stderr: 93.3% -- replaced with /var/log/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/ceph-osd.3.log.gz 2024-09-10T19:10:57.399 INFO:teuthology.orchestra.run.smithi047.stderr: 92.9% -- replaced with /var/log/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/ceph-osd.0.log.gz 2024-09-10T19:10:57.402 INFO:teuthology.orchestra.run.smithi149.stderr: 93.2% -- replaced with /var/log/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/ceph-osd.4.log.gz 2024-09-10T19:10:57.405 INFO:teuthology.orchestra.run.smithi149.stderr: 2024-09-10T19:10:57.405 INFO:teuthology.orchestra.run.smithi149.stderr:real 0m0.430s 2024-09-10T19:10:57.405 INFO:teuthology.orchestra.run.smithi149.stderr:user 0m1.599s 2024-09-10T19:10:57.405 INFO:teuthology.orchestra.run.smithi149.stderr:sys 0m0.067s 2024-09-10T19:10:57.412 INFO:teuthology.orchestra.run.smithi047.stderr: 92.8% -- replaced with /var/log/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/ceph-osd.2.log.gz 2024-09-10T19:10:57.482 INFO:teuthology.orchestra.run.smithi047.stderr: 93.1% -- replaced with /var/log/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/ceph-osd.1.log.gz 2024-09-10T19:10:57.612 INFO:teuthology.orchestra.run.smithi047.stderr: 91.0% -- replaced with /var/log/ceph/d8563d7e-6fa5-11ef-bcea-c7b262605968/ceph-mon.a.log.gz 2024-09-10T19:10:57.614 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-09-10T19:10:57.614 INFO:teuthology.orchestra.run.smithi047.stderr:real 0m0.640s 2024-09-10T19:10:57.614 INFO:teuthology.orchestra.run.smithi047.stderr:user 0m2.724s 2024-09-10T19:10:57.614 INFO:teuthology.orchestra.run.smithi047.stderr:sys 0m0.115s 2024-09-10T19:10:57.615 INFO:tasks.cephadm:Archiving logs... 2024-09-10T19:10:57.615 DEBUG:teuthology.misc:Transferring archived files from smithi047:/var/log/ceph to /home/teuthworker/archive/gabrioux-2024-09-10_17:53:37-orch:cephadm-wip-guits-quincy-2024-09-09-1810-quincy-distro-default-smithi/7900270/remote/smithi047/log 2024-09-10T19:10:57.616 DEBUG:teuthology.orchestra.run.smithi047:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-10T19:10:58.054 DEBUG:teuthology.misc:Transferring archived files from smithi149:/var/log/ceph to /home/teuthworker/archive/gabrioux-2024-09-10_17:53:37-orch:cephadm-wip-guits-quincy-2024-09-09-1810-quincy-distro-default-smithi/7900270/remote/smithi149/log 2024-09-10T19:10:58.055 DEBUG:teuthology.orchestra.run.smithi149:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-10T19:10:58.328 INFO:tasks.cephadm:Removing cluster... 2024-09-10T19:10:58.329 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d8563d7e-6fa5-11ef-bcea-c7b262605968 --force 2024-09-10T19:10:58.821 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d8563d7e-6fa5-11ef-bcea-c7b262605968 --force 2024-09-10T19:10:59.291 INFO:tasks.cephadm:Removing cephadm ... 2024-09-10T19:10:59.291 DEBUG:teuthology.orchestra.run.smithi047:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-10T19:10:59.309 DEBUG:teuthology.orchestra.run.smithi149:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-10T19:10:59.324 INFO:tasks.cephadm:Teardown complete 2024-09-10T19:10:59.325 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-10T19:10:59.334 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-10T19:10:59.334 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-10T19:10:59.352 DEBUG:teuthology.orchestra.run.smithi149:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-10T19:10:59.438 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-10T19:10:59.447 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-10T19:10:59.447 DEBUG:teuthology.orchestra.run.smithi047:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T19:10:59.450 DEBUG:teuthology.orchestra.run.smithi149:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T19:10:59.463 INFO:teuthology.orchestra.run.smithi047.stderr:bash: line 1: ntpq: command not found 2024-09-10T19:10:59.464 INFO:teuthology.orchestra.run.smithi149.stderr:bash: line 1: ntpq: command not found 2024-09-10T19:10:59.468 INFO:teuthology.orchestra.run.smithi047.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T19:10:59.468 INFO:teuthology.orchestra.run.smithi047.stdout:=============================================================================== 2024-09-10T19:10:59.468 INFO:teuthology.orchestra.run.smithi047.stdout:^* hv01.front.sepia.ceph.com 3 6 377 4 -152us[ -153us] +/- 44ms 2024-09-10T19:10:59.468 INFO:teuthology.orchestra.run.smithi047.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 5 +1285us[+1285us] +/- 64ms 2024-09-10T19:10:59.468 INFO:teuthology.orchestra.run.smithi047.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 64 -1188us[-1188us] +/- 71ms 2024-09-10T19:10:59.469 INFO:teuthology.orchestra.run.smithi047.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T19:10:59.469 INFO:teuthology.orchestra.run.smithi149.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T19:10:59.470 INFO:teuthology.orchestra.run.smithi149.stdout:=============================================================================== 2024-09-10T19:10:59.470 INFO:teuthology.orchestra.run.smithi149.stdout:^* hv01.front.sepia.ceph.com 3 6 377 1 -152us[ -165us] +/- 44ms 2024-09-10T19:10:59.470 INFO:teuthology.orchestra.run.smithi149.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 7 +1280us[+1267us] +/- 64ms 2024-09-10T19:10:59.470 INFO:teuthology.orchestra.run.smithi149.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 5 -1209us[-1222us] +/- 72ms 2024-09-10T19:10:59.470 INFO:teuthology.orchestra.run.smithi149.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T19:10:59.470 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-10T19:10:59.481 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-10T19:10:59.482 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-10T19:10:59.490 DEBUG:teuthology.orchestra.run.smithi047:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-10T19:10:59.584 DEBUG:teuthology.orchestra.run.smithi149:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-10T19:10:59.653 DEBUG:teuthology.orchestra.run.smithi047:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-10T19:10:59.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:10:59.683 DEBUG:teuthology.orchestra.run.smithi149:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-10T19:10:59.710 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:10:59.711 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-10T19:10:59.720 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-10T19:10:59.729 INFO:teuthology.task.internal:Duration was 1468.644965 seconds 2024-09-10T19:10:59.730 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-10T19:10:59.739 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-10T19:10:59.739 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-10T19:10:59.741 DEBUG:teuthology.orchestra.run.smithi149:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-10T19:10:59.779 INFO:teuthology.orchestra.run.smithi047.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T19:10:59.790 INFO:teuthology.orchestra.run.smithi149.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T19:11:00.125 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-10T19:11:00.125 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi047.front.sepia.ceph.com 2024-09-10T19:11:00.125 DEBUG:teuthology.orchestra.run.smithi047:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-10T19:11:00.147 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi149.front.sepia.ceph.com 2024-09-10T19:11:00.147 DEBUG:teuthology.orchestra.run.smithi149:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-10T19:11:00.189 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-10T19:11:00.189 DEBUG:teuthology.orchestra.run.smithi047:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-10T19:11:00.192 DEBUG:teuthology.orchestra.run.smithi149:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-10T19:11:00.255 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-10T19:11:00.255 DEBUG:teuthology.orchestra.run.smithi047:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-10T19:11:00.259 DEBUG:teuthology.orchestra.run.smithi149:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-10T19:11:00.549 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-10T19:11:00.559 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-10T19:11:00.559 DEBUG:teuthology.orchestra.run.smithi047:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-10T19:11:00.584 DEBUG:teuthology.orchestra.run.smithi149:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-10T19:11:00.611 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-10T19:11:00.621 DEBUG:teuthology.orchestra.run.smithi047:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-10T19:11:00.627 DEBUG:teuthology.orchestra.run.smithi149:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-10T19:11:00.648 INFO:teuthology.orchestra.run.smithi047.stdout:kernel.core_pattern = core 2024-09-10T19:11:00.674 INFO:teuthology.orchestra.run.smithi149.stdout:kernel.core_pattern = core 2024-09-10T19:11:00.708 DEBUG:teuthology.orchestra.run.smithi047:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-10T19:11:00.727 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:11:00.727 DEBUG:teuthology.orchestra.run.smithi149:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-10T19:11:00.763 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:11:00.763 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-10T19:11:00.773 INFO:teuthology.task.internal:Transferring archived files... 2024-09-10T19:11:00.774 DEBUG:teuthology.misc:Transferring archived files from smithi047:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/gabrioux-2024-09-10_17:53:37-orch:cephadm-wip-guits-quincy-2024-09-09-1810-quincy-distro-default-smithi/7900270/remote/smithi047 2024-09-10T19:11:00.774 DEBUG:teuthology.orchestra.run.smithi047:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-10T19:11:00.813 DEBUG:teuthology.misc:Transferring archived files from smithi149:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/gabrioux-2024-09-10_17:53:37-orch:cephadm-wip-guits-quincy-2024-09-09-1810-quincy-distro-default-smithi/7900270/remote/smithi149 2024-09-10T19:11:00.814 DEBUG:teuthology.orchestra.run.smithi149:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-10T19:11:00.849 INFO:teuthology.task.internal:Removing archive directory... 2024-09-10T19:11:00.850 DEBUG:teuthology.orchestra.run.smithi047:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-10T19:11:00.853 DEBUG:teuthology.orchestra.run.smithi149:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-10T19:11:00.898 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-10T19:11:00.911 INFO:teuthology.task.internal:Not uploading archives. 2024-09-10T19:11:00.911 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-10T19:11:00.922 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-10T19:11:00.922 DEBUG:teuthology.orchestra.run.smithi047:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-10T19:11:00.925 DEBUG:teuthology.orchestra.run.smithi149:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-10T19:11:00.938 INFO:teuthology.orchestra.run.smithi047.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 10 19:11 /home/ubuntu/cephtest 2024-09-10T19:11:00.954 INFO:teuthology.orchestra.run.smithi149.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 10 19:11 /home/ubuntu/cephtest 2024-09-10T19:11:00.955 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-10T19:11:00.973 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-10T19:11:01.007 INFO:teuthology.run:Summary data: description: orch:cephadm/with-work/{0-distro/centos_9.stream fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rotate-keys} duration: 1468.6449649333954 flavor: default owner: scheduled_gabrioux@teuthology success: true 2024-09-10T19:11:01.007 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-10T19:11:01.088 INFO:teuthology.run:pass